betaFeedback

WorksComputer Science

EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities

Abstract

Rapidly evolving LLM architectures and heterogeneous mobile hardware make supporting new models in on-device inference engines a recurring and costly system challenge. Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often miss these cross-layer dependencies and struggle to diagnose failures across host–device execution paths. We present EngineAgent, an agentic system for automating on-device LLM model enablement. Its key insight is to separate reusable engine and hardware knowledge from model-specific implementation. Before enablement, EngineAgent constructs a hierarchy-aware engine index that captures engine abstraction layers and cross-layer dependencies, and pre-validates backend-specific build, execution, and debugging workflows using already supported models. For each new model, a builder uses this knowledge to coordinate cross-layer changes, while an auditor checks implementation completeness and drives iterative debugging through device-side validation. Across 12 tasks spanning three mobile LLMs and four engine–backend pairs on CPUs, GPUs, and NPUs, EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline. Success requires compilation, on-device execution, and output-logit agreement with reference implementations. EngineAgent also reduces average per-task time and model-API cost relative to baselines using the same underlying LLMs.

Introduction

On-device LLM inference engines are critical middleware positioned between two distinct ecosystems: rapidly evolving LLM model architectures above and a fragmented landscape of heterogeneous mobile hardware below. Engine developers must continuously extend their engines to support previously unsupported model architectures and capabilities while ensuring their correct execution across underlying diverse mobile CPUs, GPUs, and NPUs.

On the model side, device-oriented lightweight LLM families evolve rapidly, with successive generations often released only months or even weeks apart. For instance, Gemma 3 to Gemma 3n only took about 4 months [12, 35], while MiniCPM-V 4.0 to 4.5 took only 24 days [44]. More importantly, successive generations frequently introduce new modules and designs. Qwen3.5 [36] introduces GatedDeltaNet linear attention [43] interleaved with conventional attention. Gemma4 [13] adds hybrid sliding-window/global attention, proportional RoPE (p-RoPE), unified keys and values, shared KV cache, and Per-Layer Embeddings (PLE). MiniCPM-V 4.6 further introduces a window-attention vit_merger, intra-ViT early compression, and mixed 4×/16× visual-token compression [26]. Consequently, supporting one generation rarely guarantees support for its successors, forcing inference engines to repeatedly incorporate newly introduced model structures and modules.

On the hardware side, mobile deployment is fragmented across heterogeneous execution backends. ARM CPUs, mobile GPUs [18], Qualcomm Hexagon NPUs [16, 17], MediaTek APUs [15], and Apple Neural Engine [14] differ substantially in programming languages and interfaces, tensor layouts, compilation workflows, and runtime semantics. Therefore, supporting the same new LLM feature requires substantially different implementation paths across hardware backends. CPU backends may implement new operators directly in C/C++ or assembly; Vulkan GPUs may require dedicated compute shaders, SPIR-V compilation, and backend-specific dispatch logic; NPUs typically require graph rewriting, accelerator-specific lowering, and vendor-specific compilation. Consequently, each newly introduced model feature need to be independently adapted and validated for multiple backends through entirely different engineering paths, multiplying the implementation efforts required for enablement.

Traditionally, enabling a new model remains a largely handcrafted, expert-driven process, requiring expertise spanning both LLM model architecture and thorough understanding of the inference engine stack. Maintainers must analyze the new model semantics, map new model features onto engine modules, modify export and runtime execution paths, adapt operators or kernels, and validate execution on target backends. For example, enabling Qwen3.5-2B in MNN [22] touched 42 files and over 4.2K lines across export and execution runtime, shape inference, and both CPU and GPU backends, whereas ExecuTorch [9] introduced changes to 20 files and over 1K lines for text-only CPU support. Such broad and heavy modifications impose substantial development and validation burdens on engine maintainers, which are also reflected in both incomplete backend and modality coverage and enablement delay. As summarized in Table 1, 2 out of the 4 on-device engines do not support Gemma4-E2B-it, while 3 out of the 4 do not support MiniCPM-V 4.6. Even when a model is supported, coverage is often restricted to only a subset of backends or modalities, such as CPU-only or text-only execution. Moreover, initial on-device enablement can take up to 30 days after model release.

ModelReleaseEngineFirst EnablementBackend
Qwen3.5-2B2026-03-02ExecuTorch04-01 / 30 daysCPU (text-only)
Qwen3.5-2B2026-03-02MNN03-02 / 0 dayCPU / GPU / NPU
Qwen3.5-2B2026-03-02llama.cpp03-02 / 0 dayCPU
Qwen3.5-2B2026-03-02MLC-LLM03-08 / 6 daysGPU
Gemma4-E2B-it2026-04-02ExecuTorchN.A.unsupported
Gemma4-E2B-it2026-04-02MNN04-09 / 7 daysCPU
Gemma4-E2B-it2026-04-02llama.cpp04-02 / 0 dayCPU
Gemma4-E2B-it2026-04-02MLC-LLMN.A.unsupported
MiniCPM-V 4.62026-05-11ExecuTorchN.A.unsupported
MiniCPM-V 4.62026-05-11MNNN.A.unsupported
MiniCPM-V 4.62026-05-11llama.cpp05-11 / 0 dayCPU
MiniCPM-V 4.62026-05-11MLC-LLMN.A.unsupported

Table 1. Model enablement2 backend coverage and delay across on-device LLM inference engines [9, 10, 22, 24].

The substantial engineering burden naturally motivates us to agentize model enablement. Before a new model is released, agents should acquire reusable knowledge of the engine’s semantics and establish backend-specific debugging and verification workflows using already supported models. After release, agents should analyze the new model’s architecture, use this knowledge to identify and implement the required engine modifications, and iteratively debug and verify the implementation on real devices.

Challenges of Agentization. However, both localizing modifications within an engine and debugging and verifying them on real devices are fundamentally challenging.

① Challenges of mapping new model features across hierarchical engine abstractions. On-device LLM engines such as llama.cpp [10], MNN [22], ExecuTorch [9], and MLC-LLM [24], share a common hierarchical organization that can be abstracted into multiple levels with distinct representations and manipulation logic. At the input level, natural-language or visual inputs are transformed through tokenization or patchification; at the graph level, tensor computation is constructed, converted, fused, and scheduled as computation graphs; and at the operator level, computations are realized through backend-specific kernels and runtime dispatch. During model enablement, a new feature may require coordinated modifications across these levels, yet their respective roles are not apparent from code structure alone. Without understanding where a component lies in the engine hierarchy and what role that level plays in model execution, an agent may identify and modify locally relevant code while overlooking modifications required at other levels. Therefore, reliable modification localization requires reasoning about the engine hierarchy rather than treating the codebase as a flat collection of files and symbols.

② Challenges of cross-device debugging and verification. Unlike conventional host-only software, where execution and debugging typically occur within the same environment, on-device model enablement follows a host-device, cross-language, build–debug pipeline spanning host-side preparation and device-side deployment and execution. As a result, a failure observed on device may originate from an earlier host-side stage rather than from the failing runtime itself. Moreover, mobile CPU, GPU, and NPU backends follow distinct compilation and runtime workflows, each requiring different debugging methods. Consequently, tracing failures back to their root causes across devices and backends is inherently challenging.

Why Existing Coding Agents Fail. Existing repository-level coding agents struggle with both challenges. ① Code structure centric knowledge rather than engine hierarchy knowledge. Existing agents [4, 6, 38, 47] primarily reason over files, functions, symbols, and AST-level dependencies without explicitly modeling how these entities related to engine abstraction hierarchy. Consequently, their reasoning and modifications are typically constrained to local changes of one or two modules, missing necessary modifications across distant abstraction layers, e.g., adding an operator definition without implementing its hardware kernel. ② Host-centric verification and debugging rather than cross-language, host-device, backend-specific workflows. Existing agents largely rely on language-specific, host-local static analysis, unit tests, and debug tracing [48, 51]. They lack comparable capabilities for orchestrating and reasoning over host–device debug workflows, causing device deployments to fail, device-side failures to remain unattributed or misattributed, and debugging to fall back to repeated build–test attempts without precise localization. Even Codex [25] powered by GPT-5.6-Sol exhibits both missing implementations and hardware-verification failures (Figure 9); for example, when enabling MiniCPM-V 4.6 on MNN–CPU, it stopped after an incomplete build and skipped device validation (Table 2).

Engine BackendAgentFailed StageFailure CategorySymptom
MNN CPUCodePlanPlanningMissing implementationLocalized config changes but missed dependent vision and runtime implementations.
MNN CPUSWE-AgentImplementationMissing implementationModified model components but missed export–runtime propagation.
MNN CPUCodexVerificationPremature acceptanceStopped after incomplete build and skipped device validation.
llama.cpp VulkanOpenHandsBackend integrationBackend failureFailed to resolve Vulkan compilation and deployment.
llama.cpp VulkanOpenCodeBackend integrationBackend failureHost adaptation completed but Vulkan execution path remained incomplete.
ExecuTorch QNNCodePlanPlanningMissing implementationFollowed local dependencies but missed QNN export.
ExecuTorch QNNOpenHandsBackend integrationBackend failureFailed at QNN lowering and runtime constraints.

Table 2. Representative failure cases of existing coding agents during on-device LLM model enablement.

Figure 9. Failure breakdown across the 12 benchmark cases.

Figure 1. Position and workflow of EngineAgent: agentically mapping rapidly evolving LLM architectures onto heterogeneous mobile hardware for inference engines.

Our System. We propose EngineAgent, an agentic system for reliable on-device LLM engine model enablement that automates the workflow while addressing the two challenges above. EngineAgent consists of three key designs: a model-agnostic engine hierarchy knowledge index, hardware-aware debug skills, and a model-specific multi-agent implementation–audit main loop. To address the lack of engine-hierarchy understanding, EngineAgent constructs a hierarchy-aware knowledge index (§ 4.2) that maps code structures onto engine abstraction layers and cross-layer dependencies, enabling agents to identify complete modification scopes beyond local code relations. To address unreliable hardware-grounded debugging, EngineAgent constructs reusable build, test, and debugging skills (§ 4.3) by validating backend-specific workflows with already supported models, providing stable verification procedures for heterogeneous hardware. These two designed artifacts are pre-constructed ahead of time because they are reusable and model-agnostic. Moreover, they are designed to be updatable upon new engine commits. When a new model arrives, it enters the model-specific multi-agent enablement main loop (§ 4.4), where newly introduced model features are mapped onto the pre-constructed hierarchy knowledge index and hardware-aware debug skills. The builder then plans and implements layer-wise modifications guided by the hierarchy index, and submits each completed change to the red-team auditor. If validation fails, failures are routed to the builder with hardware-aware debugging support for iterative refinement. The auditor accepts the implementation only after all layer-wise checks and end-to-end device validation pass.

Together, EngineAgent supports reliable model enablement across heterogeneous inference engines and hardware backends. We evaluate EngineAgent on 12 model-enablement tasks spanning 3 mobile LLMs and 4 engine–backend pairs that cover CPU, GPU, and NPU execution: MNN–CPU, llama.cpp–Vulkan, MLC-LLM–OpenCL, and ExecuTorch–QNN. EngineAgent achieves an end-to-end success rate of 75%, compared with 33.3% for the strongest coding-agent baseline.

This work makes the following contributions:

  • We identify model enablement as a distinct cross-layer software-engineering problem for on-device LLM inference engines, and characterize its two key challenges: hierarchical system understanding and hardware-aware debugging and verification across heterogeneous devices and backends.

  • We design EngineAgent, an agentic system that addresses these challenges through three key components: a model-agnostic hierarchy-aware engine knowledge index that captures abstraction layers and cross-layer dependencies, hardware-aware debug skills that provide reusable and validated build–debug workflows, and a model-specific main loop that maps new model features onto these pre-constructed artifacts for layer-wise implementation, completeness auditing, device-side validation, and iterative refinement.

  • We evaluate EngineAgent on 12 model-enablement tasks across three mobile LLMs and four engine–backend pairs, demonstrating higher effectiveness and efficiency than existing coding agents across benchmark cases.

Background

On-Device LLM Inference Engines

On-device LLM inference engines provide the unified software infrastructure for executing diverse LLMs on mobile devices equipped with heterogeneous processors such as CPUs, GPUs, and NPUs. Representative engines, including MNN [22, 39], ExecuTorch [9], llama.cpp [10], and MLC-LLM [24], share a common two-stage architecture: an offline, on-host model export stage, realized by the engine’s converter and compiler, and an online, on-device execution stage, powered by the engine’s runtime. Orthogonally, both stages are organized into the same three levels of abstraction: the input level (chat templates and tokenizers), the graph level (the computation graph and its intermediate representation, IR), and the operator level (backend kernels and their hardware mappings).

As is illustrated in Figure 2, the offline export stage traverses these three levels top-down. At the input level, it exports the chat template and the (multi-modal) tokenizer and packs them into the model artifact. At the graph level, it maps PyTorch layers and modules onto engine-native IR nodes, applying transformations such as quantization, weight packing, layout reordering, and operator fusion. At the operator level, it lowers and links each IR node to a hardware-specific kernel from the engine’s backend operator library. Notably, certain engine–hardware pairs defer preparation to runtime via online just-in-time (JIT) compilation, such as the OpenCL backend for mobile GPUs in MNN but the JIT workflow mirrors its offline counterpart level for level. The online execution stage traverses the same levels at inference time. Engine runtime loads the prepared artifact, applies input-level preprocessing and tokenization to the user input, then walks the graph node by node, dispatching each node to its operator implementation.

On-device LLM engine export and execution stages

Figure 2. On-device LLM engines comprise export and execution stages, which share three abstraction layers.

In summary, although these engines differ in their IR designs and supported backends, they converge on the same two-stage, three-level workflow. This shared abstraction is what allows our analysis to generalize across engines rather than being tied to any single instance.

Model Enablement from Maintainers’ View

For engine maintainers, supporting a new LLM in a mobile inference engine is a complicated engineering effort, involving two primary procedure: understanding the engine and mapping new model features to modifications across engine modules and abstraction layers, and perform the host-device coordinated verify-debug loop to debug. Figure 3 exemplifies these 2 major procedure.

Model enablement workloads: mapping new model features to engine abstraction layers and host-device build-verify-debug loop

Figure 3. Model enablement workloads: mapping new model features to engine abstraction layers and then perform host-device build-verify-debug loop.

Maintainers first analyze newly introduced model features and determine how they map onto the engine’s abstraction levels and modules. As illustrated by the example of mapping Qwen3.5-2B [36] to MNN [22] in Figure 3(a), changes such as new chat-template formats or multimodal settings (e.g., "<|im_start|>" and input visual patch size) are mapped to the input level; new model structures and tensor contracts such as the vision index tensor shape and data type are mapped to the graph level; newly introduced operations such as FusedLinearAttention [43] must be lowered to operator level backend specific implementations, e.g., CPUFusedLinearAttention and its ARM64 NEON implementations. After the cognitive step of these mappings from model features to abstraction layers, the maintainers implements modifications to files and modules in each layer.

Maintainers’ implementation loop is an iterative host–device verify-debug loop. As shown in Figure 3(b), they first modify on host across Python, C/C++, assembly, or backend specific code, then cross-compile it using the corresponding toolchains, such as the Android NDK, Vulkan toolchain, or QNN SDK. The resulting binaries and model artifacts are deployed to the target phone through Android Debug Bridge (ADB) for execution, after which logs, outputs, and other runtime artifacts are collected back on the host for verification. If any stage fails, maintainers trace the observed error back to the responsible implementation, revise the code, and repeat the coding–build–deploy–execute–verify cycle until all required checks pass.

Agent for Systems

To alleviate the substantial human effort in model enablement, agentizing the workflow is a natural direction. Existing works have developed complementary capabilities for software engineering: general-purpose end-to-end coding agents directly solve development tasks through iterative code modification and execution, while code-analysis tools and agents enhance end-to-end coding agents with repository representations, modification localization, and edit plans, and debugging agents augment them with failures diagnosis and generate corrective feedback. However, these capabilities do not directly resolve the challenges of inference-engine model enablement.

Limitations of General Purpose End-to-End Coding Agents. CodeAgent [47], SWE-Agent [42], and OpenHands [38] provide end-to-end workflows integrating repository navigation, code modification, execution, and iterative feedback. Recent systems efforts such as SYSSPEC [20] and ECO [19] further incorporate domain-specific abstractions and production constraints into LLM-driven system automation. However, these approaches still do not target inference-engine model enablement as an end-to-end task: they neither expose engine hierarchies as first-class knowledge nor provide dedicated host–device verification and debugging procedures across heterogeneous hardware backends.

Limitations of Prior Code Analysis Tools and Agents. CodeGraph [23] and LocAgent [6] improve repository representation and localization, CodePlan [4] plans coordinated repository-wide edits, and AutoCodeRover [48] combines program-structure-aware search with fault localization. However, these methods primarily organize knowledge around code-level structures such as files, symbols, functions, and AST or dependency relations. They do not explicitly model how these entities map onto inference-engine abstraction layers, making cross-layer dependencies, wrong-layer localization, and structurally distant but semantically coupled modifications difficult to reason about.

Limitations of Prior Debug Agents. AgenTracer [46] attributes failures to responsible agents and trajectory steps, while AgentDebug [51] identifies root causes and provides corrective feedback for recovery. Such approaches improve

failure analysis, but remain largely centered on agent trajectories or host-side program feedback. Model enablement instead requires cross-language, cross-compilation debugging across host preparation, device deployment, and heterogeneous XPU execution, where failures must be traced from device-side observations back to the responsible host-side implementation.

Case Studies and Key Insights

Characterizing Agent Failures

Motivating Case Studies. To understand the limitations of existing coding agents for on-device LLM model enablement, we conduct motivating case studies on three representative engine–backend pairs: MNN–CPU, llama.cpp–Vulkan, and ExecuTorch–QNN, with MiniCPM-V 4.6 as the target model. We evaluate representative repository-level coding agents, including SWE-Agent [42], OpenHands [38], CodePlan [4], OpenCode [27], Codex, and CodeGraph [23] + OpenCode + AgenTracer [46]. The evaluated agents use their corresponding backbone models: DeepSeek-V4-Flash for open-source agent frameworks and GPT-5.6-Sol for Codex [25].

Across these cases, existing agents demonstrate promising capability in repository exploration and local code modification, but rarely complete the entire model enablement workflow from model adaptation to validated on-device execution. Across 18 test cases (6 agents × 3 representative engine–backend pairs), only 1 attempt successfully completes end-to-end model enablement. The remaining 17 attempts fail at different stages, including cross-layer implementation, backend integration, and device-grounded verification. Notably, the only successful case, OpenCode on MNN–CPU, supports the CPU backend, which has a simpler lowering and compilation stack than GPU/NPU. These observations motivate a systematic categorization of agent failures.

Agent Failures Categorization. We summarize representative failure cases of existing coding agents in Table 2. Although the failures occur at different stages and across different engine–backend pairs, they exhibit two recurring patterns. First, agents frequently identify and modify locally relevant components but fail to recover the complete implementation scope required by a new model feature. For example, CodePlan on MNN–CPU modified model-related configuration paths but missed the dependent vision and runtime implementations, while SWE-Agent modified model components without propagating changes across export and execution paths. Second, agents often achieve intermediate host-side progress but fail to complete hardware-grounded validation. For example, OpenCode and OpenHands on llama.cpp–Vulkan reached host-side adaptation but failed to establish the complete Vulkan execution workflow, while Codex prematurely stopped after an incomplete build state without reaching device validation. These failure patterns indicate that reliable model enablement requires both engine-level understanding across abstraction layers and hardware-grounded verification capabilities.

Key Insights

The observed failures reveal two fundamental requirements for reliable agentic model enablement: global understanding of inference-engine abstractions and stable hardware-grounded verification workflows.

Insight 1: Model Enablement Agent Requires Engine Hierarchy Aware Understanding. Missing implementations and premature exits often stem from agents lacking a global view of the inference engine organization. Without understanding the responsibilities and dependencies of different abstraction layers, agents may treat locally discovered code paths as the complete implementation scope and overlook required modifications elsewhere. Therefore, agents require explicit hierarchy-aware engine knowledge to map model features to affected layers, recover complete modification scopes, and guide cross-layer implementation.

Insight 2: Model Enablement Agent Requires Hardware Aware Debug Ability. Hardware-aware debugging is challenging not only because device execution is heterogeneous, but also because the verification path itself may be unstable when enabling previously unsupported models. Agents that directly debug unsupported models must simultaneously resolve model adaptation issues and uncertain build, deployment, and validation workflows, making failures difficult to attribute. Therefore, reliable enablement requires pre-constructed and validated build, test, and debugging workflows from already supported models, which provide stable hardware-grounded feedback for subsequent model adaptation.

System Design

System Overview

Guided by the two key insights, EngineAgent augments general-purpose coding agents with two capabilities: engine-hierarchy-aware understanding for complete cross-layer implementation and hardware-aware debugging and verification for reliable on-device validation. These capabilities support three complementary agent roles: a builder that implements model support, a debugger that diagnoses failures, and an independent auditor that challenges implementation completeness and correctness.

Following Insight 1, EngineAgent equips agents with a model-agnostic engine hierarchy knowledge index (§4.2). The index connects engine abstractions to concrete code entities and exposes dependencies across export, runtime, and backend layers. It guides the builder in mapping new model features to the required modification scope and provides the auditor with structural context for identifying omitted implementations. Following Insight 2, EngineAgent equips agents with reusable hardware-aware verify-debug skills (§4.3). Before a new model arrives, construction agents establish and validate backend-specific build, deployment, testing, and debugging workflows using already supported models. These skills provide the builder with self-validation procedures, the debugger with diagnostic guidance, and the auditor with executable checks, reducing uncertainty in the verification path during new-model adaptation.

As illustrated in Figure 4, both capabilities are prepared before model release and reused in the model-specific multi-agent main loop (§4.4). When a new model arrives, the builder compares its architecture with supported models, maps the differences onto the hierarchy index, and implements layer-wise modifications. The auditor checks each completed layer, while the debugger uses the validated host–device workflows to diagnose failures and guide revisions. The loop continues until the implementation passes both layer-wise checks and end-to-end device validation.

Two-stage workflow of EngineAgent, from engine-specific knowledge construction to model-dependent enablement and verification

Figure 4. Two-stage workflow of EngineAgent, from engine-specific knowledge construction to model-dependent enablement and verification.

Hierarchy-Aware Engine Understanding

For a fixed engine snapshot, prior work CodeGraph [23] provides a language-agnostic canonical graph of files, symbols, and typed static relations, including imports, calls, references, and instantiations, as illustrated in Figure 5. However, it captures only how code entities are structurally connected, without revealing the abstract knowledge of engine hierarchical design.

CodeGraph and abstraction hierarchy diagram

Figure 5. CodeGraph [23] only captures syntactic structures rather than hierarchical understandings. Extracted relations in MNN CodeGraph as an example are illustrated on the left.

To encounter this, EngineAgent defines a nine-layer hierarchy based on the design abstraction of on-device LLM engines and automatically constructs a hierarchy-aware index by semantically lifting the CodeGraph representation of the engine into our pre-defined hierarchy. In this way, engine abstractions are explicitly represented as structured semantic subgraph views over the original CodeGraph, extending syntactic relations to system-level understanding.

Hierarchy Abstraction. As discussed in § 2.1, on-device LLM engines follow a two-stage workflow, where each stage contains three-level: input level, graph-level, and operator-level. In order to better help agents understand the engine hierarchy, we further divide the three levels into nine layers, including ① chat template and tokenization and ② multimodal preprocessing layers on input level; ③ model configuration, ④ I/O tensor contracts, ⑤ module mapping, ⑥ weight sharing and KV sharing layers on graph level; and ⑦ attention and KV cache, ⑧ operator wrapping, and ⑨ operator kernel layers on operator level. Specifically, the offline-export has eight of these layers (L1–L8), spanning the export or compilation of each layer. The execution-stage, on the other hand, contains all nine abstraction layers (L1–L9), extending the export-stage hierarchy with an additional L9 layer for hardware-specific kernels, covering the parsing and execution of exported artifacts on each layer. For each abstraction layer, engine may have one or more instantiations: model-related layers (L3/L4) distinguish text-backbone and multimodal instantiations, while backend-dependent layers (offline L7–L8 and runtime L7–L9) are instantiated separately for CPU, Vulkan, OpenCL, and QNN when applicable. We define these intra-layer substructures as layer instantiations. By defining this layered hierarchy in prompt, EngineAgent

agentically construct a knowledge index from the raw CodeGraph to incorporate these knowledge into graph structures.

Agentic Index Construction and Review. EngineAgent constructs the hierarchy index as a nested graph structure following the predefined abstraction hierarchy. The construction agent first divides the raw CodeGraph into export and execution stage subgraphs and then sequentially builds abstraction-layer subgraphs within each stage. For each layer, the agent builds semantic understanding from the predefined hierarchy knowledge and repository evidence, selects representative code entities as seeds, and initializes the subgraph. The subgraph is then expanded from these seeds by traversing CodeGraph relations and incorporating entities supported by additional evidence. To improve reliability, EngineAgent separates semantic hierarchy attribution from raw graph construction, where static relation errors are checked independently while layer membership is determined through semantic evidence. After construction, the indexing agent performs self-review and a read-only auditor validates the completeness and grounding of the generated subgraphs. The resulting index provides multiple semantic views of the code entity graph across stages, abstraction layers, and instantiations. By mapping engine functional abstractions onto concrete code entities, it bridges the gap between repository structure and engine organization, enabling agents to understand how high-level functionalities are realized by specific modules.

Example and Analysis (MNN Hierarchy Indexing). Figure 6 visualizes the hierarchy-aware graph index constructed for MNN. The middle panel shows the original CodeGraph, where files and symbols are connected only through static code relations. After semantic lifting, EngineAgent reorganizes these entities into export- and execution-stage subgraphs and further into abstraction-layer subgraphs. The resulting layers exhibit clear semantic coherence by grouping code entities according to their engine responsibilities.

Hierarchy-aware engine modeling for MNN

Figure 6. Hierarchy-aware engine modeling for MNN. The original CodeGraph (center) is organized into abstraction-layer subgraphs for export (top, L1–L8) and execution (bottom, L1–L9), with layer definitions shown on the left. Colored subgraphs group code entities by engine responsibility, while gray edges indicate cross-layer dependencies.

For the export stage, L1 groups tokenization-related entities such as Tokenizer export, Normalizer, and Vocabulary; L2 captures multimodal preprocessing components such as Resampler; L3 organizes model initialization logic including Model config and Weight loading; L5 groups module-level transformations such as LayerNorm and Linear; L7 captures attention-related components including Attention, KV cache, and Rotary position; and L8 connects high-level operators with their lowered implementations, such as mapping Attention to Fuse attention. The execution-stage hierarchy further exposes corresponding runtime responsibilities: L1 contains runtime token processing entities such as BPE encoder and special token mapping; L2 includes vision feature processing and multimodal fusion; L5 groups runtime execution logic including module forward, graph traversal, and memory management; L7 captures fused attention kernels and native flexible KV cache implementations; L8 contains concrete operators such as Convolution and QNN MatMul; and L9 reaches hardware-specific implementations such as ARM INT4 pack, I8MM, and SME2 GEMM. These examples demonstrate that the generated hierarchy is not a syntactic partition of the repository, but a semantic mapping between engine functionalities and concrete code entities. It enables agents to reason about how high-level model functionalities are implemented across different abstraction layers.

Abstraction-layer subgraphs with multiple concrete realizations are further refined into subordinate instantiation subgraphs. As shown in Figure 7, MNN’s L3 separates backbone and multimodal implementations, while L8 branches into CPU, Vulkan, OpenCL, and QNN implementations with distinct backend-specific code paths. These instantiation subgraphs expose how the same engine abstraction is realized differently across modalities and hardware backends.

Figure 7. Instantiation subgraphs within MNN’s execution-stage hierarchy, exposing modality- and backend-specific code structures within each layer.

Host–Device Debugging and Verification Skills

Model enablement requires not only modifying the correct engine layers, but also repeatedly building, deploying, executing, and diagnosing the resulting implementation on real devices. To support reliable hardware-aware verify-debug loop, EngineAgent agentically constructs dedicated verify-debug skills around these host–device workflows. These procedures are largely engine and backend specific, yet remain reusable across subsequent new model enablements.

Automatic Skill Construction. To provide agents with procedural knowledge for backend-specific building, verification, and failure diagnosis, EngineAgent agentically constructs build skills, unit tests, and few-shot debug skills before individual model-enablement tasks.

Build skills organize information scattered across documentation, build scripts, and repository instructions into two executable workflows: model conversion/export and execution runner build and deployment. Rather than merely summarizing documentation, the construction agent explores the relevant scripts and commands and validates the resulting workflow by exporting and executing an already supported model on the real device.

Besides, unit tests and few-shot debug skills are further constructed following the layer and instantiation division defined in hierarchy-aware graph index. For the export stage, it traverses L1–L8 in order and builds tests around the observable input/output contracts of each abstraction layer, from preprocessing and model configuration to module mapping and operator lowering. For the execution stage, it similarly traverses L1–L9, extending validation through runtime operators and hardware-specific kernels. When a layer contains multiple instantiations, tests are constructed separately for the corresponding modality or backend branches. Figure 8 illustrate the example of generated skills for MNN-CPU execution stage. These generated skills check and verify the outputs and functionality of each layer’s modules. Together, these tests form a progressive validation chain, allowing failures to be localized to the earliest layer whose contract is violated rather than only being observed at end-to-end execution. During this construction process, successful diagnostic procedures, characteristic failure symptoms, and corresponding recovery actions are summarized into few-shot debug skills associated with their abstraction layers and hardware backends.

Update Skills under Engine Modification. Prebuilt hardware aware verify-debug skills are updated using the same incremental principle as engine knowledge index updates. Changes to build scripts, documentation, backend implementations, or toolchain configurations invalidate only their associated build skills, unit tests, and few-shot debug skills. These affected artifacts are re-examined and revalidated, including replaying changed build or deployment workflows with an already supported model when necessary, while retaining unaffected artifacts. A complete reconstruction is performed only when the corresponding backend workflow or toolchain undergoes substantial changes.

Model-Specific Multi-Agent Main Loop

When a new model arrives, EngineAgent uses the prebuilt hierarchy index and debug skills to modify the target engine for a specified XPU backend. This process is illustrated in Figure 4. EngineAgent instantiates three model-specific agents from the pre-constructed artifacts: a builder agent for implementation, a debugger agent for debugging reinforcement, and an independent auditor agent for red-team verification.

The builder agent first performs a difference analysis against the engine’s supported model set constructed during pre-enablement to identify the most similar reference models. The architectural differences between the reference and new models are mapped onto the hierarchy-aware graph index to derive layer-wise modification plans. The builder then implements these tasks sequentially across abstraction layers. For each layer, the builder retrieves relevant hierarchy knowledge and hardware-aware build skills, unit tests, and few-shot debug skills to guide implementation and self-validation.

The debugger agent reinforces the builder when failures occur or progress becomes blocked. Given the current abstraction layer, backend, execution logs, and validation results, it retrieves corresponding debugging skills, reproduces relevant workflows when necessary, and provides targeted diagnostic evidence and recovery suggestions. By leveraging pre-constructed debug skills, the debugger avoids repeatedly exploring unstable verification paths during enablement.

The auditor agent independently evaluates the builder’s implementation as a red-team verifier. After each abstraction layer is completed, it executes the corresponding pre-constructed unit tests and checks whether the implementation satisfies layer-specific requirements. If validation fails, the auditor returns the earliest failing layer and device-side evidence for subsequent debugging. After all layers pass, the auditor performs end-to-end validation covering model export, deployment, and on-device execution before approving the implementation.

Together, these agents form an iterative implementation–verification loop. The builder proposes modifications, the auditor challenges their correctness, and the debugger provides targeted recovery when failures occur. The loop continues until all layer-wise checks and end-to-end device validation pass. Finally, the auditor-approved implementation is submitted to a rule-based non-agentic system verifier, which compares output logits against host-side PyTorch references and rejects implementations whose deviations exceed a pre-defined numerical tolerance.

Evaluation

Benchmark Design

To evaluate the effectiveness of agentic systems for on-device LLM model enablement, we construct a benchmark comprising 12 tasks across four representative engine–backend pairs and three recent mobile LLMs: Qwen3.5-2B [36], Gemma4-E2B-it [13], and MiniCPM-V 4.6 [26]. Each task requires enabling one target model on one engine–backend pair, covering diverse model architectures and heterogeneous CPU, GPU, and NPU execution paths.

Engine–Backend Pairs. The benchmark covers four representative mobile engine–backend pairs:

  • MNN + CPU,

  • llama.cpp + Vulkan,

  • MLC-LLM + OpenCL,

  • ExecuTorch + QNN.

These pairs span substantially different execution stacks and hardware abstractions. MNN + CPU [22] represents a native CPU execution path with direct operator and kernel implementations; llama.cpp + Vulkan [10] represents explicit GPU execution through Vulkan kernels and backend dispatch; MLC-LLM + OpenCL [24] captures a compiler-based deployment workflow targeting mobile GPUs; and ExecuTorch + QNN [9] represents graph lowering and execution through a vendor-specific NPU backend. Together, they cover CPU, GPU, and NPU execution as well as native, compiler-based, and accelerator-specific enablement workflows.

Benchmark Case Construction and Engine Snapshot Selections. For each model–engine–backend pair, if the target model has already been supported, we reset the engine to the commit immediately preceding the support commit and ask the agent to reproduce the enablement from this pre-support state. Under this setting, the agent is not allowed to access future commits or search the web for post-snapshot implementations. If the target model is not yet supported by the corresponding engine–backend pair, we use the latest available engine snapshot and construct a forward-enable task. Such tasks reflect realistic maintenance scenarios where no ground-truth support patch exists and success can only be determined through end-to-end validation. Each benchmark task provides the agent with an engine repository snapshot, a target model checkpoint, the backend configuration, and validation scripts. This setup requires the agent to solve the actual model-enablement problem rather than recover an existing implementation.

Success Evaluation and Criteria. A task is considered successful only if the upgraded engine passes host compilation, device execution, and output-logit alignment checks verified by our evaluation program. The evaluation program reproduces the provided build scripts to verify model export and runner cross-compilation, then deploys the generated artifacts to the target phone through ADB and checks that execution completes without errors. It subsequently runs multiple on-device test cases covering all required modalities, collects the output logits, and compares them against host-side PyTorch reference results. The implementation is accepted only when the logits agree with the reference outputs, with only negligible numerical drift attributed to hardware differences.

Experimental Setup

Baselines. We compare against four representative coding-agent baselines. SWE-Agent [42] represents repository-level autonomous software engineering, while OpenCode [27] and Codex [25] represent general-purpose coding agents. We further construct a composite baseline, OpenCode + CodeGraph + AgenTracer, augmenting OpenCode with graph-based repository understanding [23] and trajectory-level failure diagnosis [46]. Since coding agents frequently terminate prematurely on long-running enablement tasks without human continuation prompts, we enable the goal mode of Codex and attach a lightweight continuous-run trigger to the other baselines, which resumes the agents when the evaluator has not yet accepted the implementation as successful and the time limit has not been reached.

Agent Model API. SWE-Agent, OpenCode, and OpenCode + CodeGraph + AgenTracer use the DeepSeek-V4-Flash API [8], while Codex uses GPT-5.6 Sol [34]. We evaluate EngineAgent with both DeepSeek-V4-Flash and GPT-5.6 Sol. This provides matched comparisons against both groups of baselines and separates the effectiveness of the EngineAgent harness from LLM API calls.

Metrics. We primarily report end-to-end success rate, together with five stage-wise success rates: export, host text-only, host multimodal, device text-only, and device multimodal, to localize failures along the enablement pipeline. A stage is counted as successful only when all associated evaluator checks described above pass. We also report average wall-clock time and average monetary cost per task. Each run is capped at five hours, with timeouts counted as five hours; monetary cost is measured by the total model-API usage incurred during the run.

Overall Performance across Benchmark Cases

Table 3 summarizes performance across 12 model-enablement tasks. EngineAgent achieves 9/12 end-to-end successes (75%) with either DeepSeek-V4-Flash or GPT-5.6 Sol, substantially outperforming all baselines. The strongest baseline, Codex with GPT-5.6 Sol, succeeds on only 4/12 tasks (33.3%), while OpenCode and CodeGraph + OpenCode + AgenTracer each succeed on 1/12 and SWE-Agent on none.

Agent Model APIAgentic SystemsE2E Success RateAverage TimeCost ($)
DeepSeek V4 FlashSWE-Agent0/124.85h2.11
DeepSeek V4 FlashOpenCode1/124.78h2.28
DeepSeek V4 FlashCodeGraph + OpenCode + AgenTracer1/124.91h2.44
DeepSeek V4 FlashEngineAgent9/123.82h1.53
GPT-5.6 SolCodex4/124.82h14.90
GPT-5.6 SolEngineAgent9/124.16h11.29

Table 3. Overall model enablement performance across the 12 benchmark cases.

EngineAgent achieves higher success rate while using less time and lower model-API cost than the baselines. With DeepSeek-V4-Flash, EngineAgent completes each task in 3.82 hours on average at a cost of $1.53, compared with 4.78–4.91 hours and $2.11–$2.44 for the corresponding baselines. With GPT-5.6 Sol, EngineAgent averages 4.16 hours and $11.29 per task, compared with 4.82 hours and $14.90 for Codex. Table 4 shows where the performance gap emerges along the enablement pipeline. With DeepSeek-V4-Flash, EngineAgent achieves 10/12 successes in export and host-side text-only and multimodal execution, and 9/12 on both device-side paths. In contrast, DeepSeek-based baselines reach at most 4/12 at the host stage and fall to 0–1/12 on device. Adding CodeGraph and AgentTracer to OpenCode improves host-side multimodal success from 2/12 to 4/12, but device-side success remains 1/12, suggesting that stronger repository understanding alone is insufficient for hardware-grounded execution. With GPT-5.6 Sol, Codex reaches 6/12 on export and host text-only execution and 4/12 on multimodal execution, while EngineAgent still achieves 10/12 on host and 9/12 on device. Overall, EngineAgent’s advantage extends beyond finding the correct implementation to maintaining correctness through multimodal integration and host-to-device execution.

Agent Model APIAgentic SystemsExportHost THost MMDevice TDevice MM
DeepSeek V4 FlashSWE-Agent4/123/120/121/120/12
DeepSeek V4 FlashOpenCode4/124/122/121/121/12
DeepSeek V4 FlashCodeGraph + OpenCode + AgentTracer4/124/124/121/121/12
DeepSeek V4 FlashEngineAgent10/1210/1210/129/129/12
GPT-5.6 SolCodex6/126/124/126/124/12
GPT-5.6 SolEngineAgent10/1210/1210/129/129/12

Table 4. Stage-wise model enablement success across the 12 benchmark cases. (T: Text-only backbone success rate on host/device, MM: Multi-Modal success rate on host/device)

Figure 9 breaks down failures across the 12 tasks. SWE-Agent and OpenCode each miss three required implementations and incur five graph-level failures, while Codex still fails in eight cases across all four categories. In contrast, EngineAgent eliminates both missing-implementation and hardware deployment/verification failures, and reduces hardware-kernel failures to 1/12. This suggests that hierarchy-aware knowledge helps recover the required implementation scope, while a stable build–verify–debug workflow prevents deployment failures and provides reliable feedback for hardware-specific debugging. As a result, EngineAgent’s remaining three failures are limited to two graph-level and one hardware-kernel implementation failure.

Breakdown Analysis By Cases

Case Study: Enabling Qwen3.5-2B on llama.cpp-Vulkan. The key challenge in enabling Qwen3.5-2B on llama.cpp-Vulkan is supporting the newly introduced Linear Attention operator on the Vulkan backend. As illustrated in Figure 10, SWE-Agent fails at the graph-export stage because it overlooks an unsupported implementation required by the new operator. OpenCode, CodeGraph + OpenCode + AgentTracer, and Codex progress further but eventually fail when implementing the execution-stage Linear Attention kernel. In particular, these agents struggle to localize kernel-level implementation errors merely with their continuous end-to-end verification. In contrast, EngineAgent exploits its hardware-aware build-test-debug skills and directly constructs unit tests for the Linear Attention kernel and iteratively debugs the implementation on device. This hardware-grounded verification loop enables EngineAgent to progressively localize and fix the implementation errors, ultimately completing the enablement in approximately 4.5 hours.

Figure 10. Progress Comparison of Agents enabling Qwen3.5-2B in llama.cpp-Vulkan.

Case Study: Enabling MiniCPM-V 4.6 on ExecuTorch-QNN. The key challenge in enabling MiniCPM-V 4.6 on ExecuTorch-QNN is to jointly modify the host-side offline QNN graph compilation and the online execution runtime. As illustrated in Figure 11, all baselines fail during export: they repeatedly modify and debug the graph and operator export paths, but eventually time out because each iteration incurs the high latency of offline QNN preparation. In contrast, EngineAgent leverages its hierarchy-aware index to directly localize the required changes for the newly introduced window-attention vit_merger in the Python exporter, successfully exporting the model within two attempts and completing the enablement at last.

Figure 11. Progress Comparison of Agents enabling MiniCPM-V 4.6 in ExecuTorch-QNN.

Cost Analysis for Pre-Constructed Index and Skills

The engine hierarchy-aware index and hardware-aware skills are pre-constructed once and reused across model-enablement tasks. As engines evolve, they are incrementally updated only the parts that are affected by each commit. We evaluate both their initial construction and incremental update costs below.

Initial Construction Costs. Table 5 reports the initial construction costs for the four evaluated engines. Constructing the hierarchy index costs 0.66−0.66-1.25,whilehardware−awareskillscost, while hardware-aware skills cost1.83–2.682.68, averaging 0.930.93 and 2.302.30, respectively. Thus, the reusable pre-enablement artifacts require only modest one-time construction cost.

EngineSnapshotIndex Cost ()∣SkillsCost() | Skills Cost ()
MNNf41d46381.041.83
llama.cppe06088da0.662.27
MLC-LLM840506970.772.40
ExecuTorchb77815521.252.68

Table 5. Initial construction cost of pre-enablement artifacts on four engine snapshots.

Update Costs. Table 6 reports the incremental cost across five consecutive MNN updates. On average, updating the hierarchy index costs only 0.00770.0077 and updating the hardware-aware skills costs 0.06460.0646 per commit, corresponding to only 0.7% and 3.5% of their initial construction costs, respectively.

Commit UpdateIndex Update Cost ()∣SkillsUpdateCost() | Skills Update Cost ()
f41d4638 → c59030820.00410.0000
c5903082 → 0bff03cb0.01950.1262
0bff03cb → 17c00ba60.00510.0000
17c00ba6 → 4419e33b0.00470.0507
4419e33b → 30dfb9bb0.00490.1459
Average0.00770.0646

Table 6. Update cost of pre-constructed artifacts across MNN versions.

Combined, each update costs only 0.0720.072 on average, about 2.5% of the initial pre-construction cost, demonstrating that the reusable artifacts can be maintained with very low overhead as the engine evolves.

Ablation Study

We ablate the two pre-constructed components of EngineAgent to quantify their contributions. As shown in Figure 12, removing the hierarchy-aware index reduces end-to-end success from 9/12 to 3/12 and increases average exploration turns from 42 to 271, showing that explicit hierarchy knowledge is critical for recovering complete implementation scopes and cross-layer dependencies. Removing the hardware-aware skills reduces success from 9/12 to 5/12 and increases compilation/debugging failures from 11 to 83, highlighting the importance of pre-validated build, verification, and debugging workflows for reliable hardware-grounded execution. Together, these results show that the two components are complementary: hierarchy-aware indexing guides complete implementation, while hardware-aware skills enable reliable on-device validation and debugging.

Ablation study of the hierarchy-aware index and hardware-aware skills

Figure 12. Ablation study of the hierarchy-aware index and hardware-aware skills.

Related Works

General-Purpose Coding Agents

Commercial coding agents [1, 3, 11, 25], SWE-agent [42], and OpenHands [38] integrate repository exploration, editing, and execution feedback. Agentless [40] structures issue resolution into localization, repair, and validation. Repository understanding draws on dependency-aware planning [4], code graphs [6, 21, 29], and program structure [48]. Other work supports interactive debugging [45], reusable skills learned from execution traces [41], and tree search with agent-based evaluation [2].

Mobile Programming and Deployment Agents

AutoIOT [32] and GPIOt [30] synthesize AIoT programs from natural-language requirements, while AgentDeploy [31] partitions models across existing inference engines and reuses optimization configurations for mobile deployment. At the operator level, MoKA, introduced in MobileKernelBench [52], combines repository-aware reasoning with iterative implementation and on-device validation. EngineAgent targets complete support for previously unsupported LLM and multimodal architectures, requiring coordinated changes across model conversion, graph construction, runtime-state management, and backend execution, validated through end-to-end execution on the target device.

Automation of ML Systems

TVM [5], Ansor [49], Roller [50], and Welder [33] automate tensor compilation and optimization through graph transformations, scheduling, search, and fusion. KernelBench [28] evaluates LLM-generated GPU kernels, while GEAK [37] and KernelAgent [7] guide kernel optimization with execution feedback and hardware information.

Conclusion

Supporting rapidly evolving LLMs on heterogeneous mobile hardware makes model enablement a recurring cross-layer systems challenge. We presented EngineAgent, which separates reusable engine and hardware knowledge from model-specific implementation. Its hierarchy-aware index guides coordinated changes across engine abstractions, while hardware-aware workflows and a builder–auditor loop ground implementation and debugging in device-side validation. Our evaluation shows that this design improves model-enablement success while reducing time and model-API cost compared with coding-agent baselines. More broadly, EngineAgent offers a path to turn specialized integration expertise into reusable infrastructure, reducing the gap between model innovation and deployment across diverse hardware.

References

  1. [1]Anthropic. 2026. Claude Code. https://www.anthropic.com/claude-code. Accessed: 2026-09-02.
  2. [2]Antonis Antoniades, Albert Orwall, Kexun Zhang, Yuxi Xie, Anirudh Goyal, and William Yang Wang. 2025. SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. https://openreview.net/forum?id=G7sIFXugTXarxiv.org/abs/2410.20285
  3. [3]Anysphere, Inc. 2026. Cursor. https://cursor.com/. Accessed: 2026-09-02.
  4. [4]Ramakrishna Bairi, Atharv Sonwane, Aditya Kanade, Vageesh D C, Arun Iyer, Suresh Parthasarathy, Sriram Rajamani, B. Ashok, and Shashank Shet. 2024. CodePlan: Repository-level Coding using LLMs and Planning. Proceedings of the ACM on Software Engineering 1, FSE (July 2024), 675–698.arxiv.org/abs/2309.12499
  5. [5]Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, and Arvind Krishnamurthy. 2018. TVM: an automated end-to-end optimizing compiler for deep learning. In Proceedings of the 13th USENIX Conference on Operating Systems Design and Implementation (OSDI ’18). USENIX Association, Carlsbad, CA, USA, 579–594.arxiv.org/abs/1802.04799
  6. [6]Zhaoling Chen, Robert Tang, Gangda Deng, Fang Wu, Jialong Wu, Zhiwei Jiang, Viktor K. Prasanna, Arman Cohan, and Xingyao Wang. 2025. LocAgent: Graph-Guided LLM Agents for Code Localization. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 - August 1, 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher H. P. 2025. Association for Computational Linguistics, 8697–8727. https://doi.org/10.18653/V1/2025.ACL-LONG.426
  7. [7]Kaiming Cheng, Laura Wang, Jack Khuu, Mark Saroufim, Wenyan Chi, Jiannan Wang, and Joe Jaacson. 2026. KernelAgent: Hardware-Guided GPU Kernel Optimization via Multi-Agent Orchestration. PyTorch Blog. https://pytorch.org/blog/kernelagent-hardware-guided-gpu-kernel-optimization-via-multi-agent-orchestration/ Published: 2026-03-06. Accessed: 2026-09-25.
  8. [8]DeepSeek-AI, Anyi Xu, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, et al. 2026. DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence. arXiv:2606.19348 [cs.CL] https://arxiv.org/abs/2606.19348
  9. [9]PyTorch Foundation. 2025. GitHub - pytorch/executorch: On-device AI across mobile, embedded and edge for PyTorch. https://github.com/pytorch/executorch.
  10. [10]Georgi Gerganov. 2026. GitHub - ggml-org/llama.cpp: LLM inference in C/C++. https://github.com/ggml-org/llama.cpp.
  11. [11]GitHub. 2026. GitHub Copilot. https://github.com/features/copilot. Accessed: 2026-09-02.
  12. [12]Google DeepMind. 2025. Gemma 3n. https://deepmind.google/models/gemma/gemma-3n/. Accessed: 2026-09-09.DOI
  13. [13]Google DeepMind. 2026. Gemma 4. https://deepmind.google/models/gemma/gemma-4/. Accessed: 2026-06-30.
  14. [14]Apple Inc. 2026. CoreML Documentation. https://developer.apple.com/documentation/coreml.
  15. [15]MediaTek Inc. 2026. NeuroPilot Documentation. https://neuropilot-developer.mediatek.com/sphinx/neuropilot-8-public/html/.
  16. [16]Qualcomm Technologies Inc. 2026. Qualcomm Documentation — HTP Backend. https://docs.qualcomm.com/doc/80-63442-10/topic/htp_backend.html. [Accessed 25-02-2026].
  17. [17]Qualcomm Technologies Inc. 2026. Qualcomm Documentation — Introduction to NPU. https://docs.qualcomm.com/nav/home/hexagon-dsp-sdk-introduction-to-npu.html?product=1601111740010422. [Accessed 25-02-2026].
  18. [18]Arm Limited. 2020. The Bifrost Shader Core Version 1.0. https://developer.arm.com/documentation/102546/0100/?lang=en.
  19. [19]Hannah Lin, Martin Maas, Maximilian Roquemoore, Arman Hasanzadeh, Fred Lewis, Yusuf Simonson, Ameya Shringi, Hongwen Dai, Patrick Musau, Tzu-Wei Yang, Amir Yazdanbakhsh, Deniz Altınbüken, Florin Papa, Maggie Nolan Edmonds, Aditya Patil, Don Schwarz, Satish Chandra, Chris Kennelly, Milad Hashemi, and Parthasarathy Ranganathan. 2026. ECO: An AI-Driven Code Efficiency Optimizer for Warehouse Scale Computers (Operational Systems). In 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI 26). USENIX Association, Seattle, WA, 589–625.
  20. [20]Qingyuan Liu, Mo Zou, Hengbin Zhang, Dong Du, Yubin Xia, and Haibo Chen. 2026. Sharpen the Spec, Cut the Code: A Case for Generative File System with SYSSPEC. In 24th USENIX Conference on File and Storage Technologies (FAST 26). USENIX Association, Santa Clara, CA, 291–311.arxiv.org/abs/2512.13047
  21. [21]Xiangyan Liu, Bo Lan, Zhiyuan Hu, Yang Liu, Zhicheng Zhang, Fei Wang, Michael Qizhe Shieh, and Wenmeng Zhou. 2025. CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025 - Volume 1: Long Papers, Albuquerque, New Mexico, USA, April 29 - May 4, 2025, Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, 142–160. https://doi.org/10.18653/V1/2025.NAACL-LONG.7
  22. [22]Chengfei Lv, Chaoyue Niu, Renjie Gu, Xiaotang Jiang, Zhaode Wang, Bin Liu, Ziqi Wu, Qiulin Yao, Congyu Huang, Panos Huang, Tao Huang, Hui Shu, Jinde Song, Bin Zou, Peng Lan, Guohuan Xu, Fei Wu, Shaojie Tang, Fan Wu, and Guihai Chen. 2022. Walle: An End-to-End, General-Purpose, and Large-Scale Production System for Device-Cloud Collaborative Machine Learning. In Proceedings of USENIX Symposium on Operating Systems Design and Implementation (OSDI ’22). USENIX, Carlsbad, CA, USA, 249–265.arxiv.org/abs/2205.14833
  23. [23]Colby McHenry. 2026. CodeGraph. https://github.com/colbymchenry/codegraph. Accessed: 2026-09-02.
  24. [24]MLC team. 2023-2025. MLC-LLM. CMU Foundation and Language Model Center. https://github.com/mlc-ai/mlc-llm
  25. [25]OpenAI. 2026. Codex. https://openai.com/codex/. Accessed: 2026-09-02.
  26. [26]OpenBMB. 2026. MiniCPM-V 4.6. https://huggingface.co/openbmb/MiniCPM-V-4.6. Hugging Face model card; accessed: 2026-06-30.
  27. [27]OpenCode Contributors. 2026. OpenCode: The Open Source Coding Agent. https://github.com/anomalyco/opencode. Accessed: 2026-09-02.
  28. [28]Anne Ouyang, Simon Guo, Simran Arora, Alex L Zhang, William Hu, Christopher Re, and Azalia Mirhoseini. 2025. KernelBench: Can LLMs Write Efficient GPU Kernels?. In Forty-second International Conference on Machine Learning (ICML ’25).arxiv.org/abs/2502.10517
  29. [29]Siru Ouyang, Wenhao Yu, Kaixin Ma, Zilin Xiao, Zhihan Zhang, Mengzhao Jia, Jiawei Han, Hongming Zhang, and Dong Yu. 2025. RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24–28, 2025. OpenReview.net. https://openreview.net/forum?id=dw9VUsSHGBarxiv.org/abs/2410.14684
  30. [30]Leming Shen, Qiang Yang, Xinyu Huang, Zijing Ma, and Yuanqing Zheng. 2025. GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Development. In Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems (UC Irvine Student Center, Irvine, CA, USA) (SenSys ’25). Association for Computing Machinery, New York, NY, USA, 199–212.arxiv.org/abs/2503.00686
  31. [31]Leming Shen, Qiang Yang, Xinyu Huang, Zijing Ma, Yuanqing Zheng, and Chris Xiaoxuan Lu. 2026. Automating Model Deployment on Mobile Devices. In Proceedings of the 24th Annual International Conference on Mobile Systems, Applications and Services Workshops (University of Cambridge, Cambridge, United Kingdom) (MobiSys Workshop ’26). Association for Computing Machinery, New York, NY, USA, 188–194.DOI
  32. [32]Leming Shen, Qiang Yang, Yuanqing Zheng, and Mo Li. 2025. Au-IOT: LLM-Driven Automated Natural Language Programming for AIoT Applications. In Proceedings of the 31st Annual International Conference on Mobile Computing and Networking (Kerry Hotel, Hong Kong, Hong Kong, China) (ACM MobiCom ’25). Association for Computing Machinery, New York, NY, USA, 468–482.
  33. [33]Yining Shi, Zhi Yang, Jilong Xue, Lingxiao Ma, Yuqing Xia, Ziming Miao, Yuxiao Guo, Fan Yang, and Lidong Zhou. 2023. Welder: Scheduling Deep Learning Memory Access via Tile-graph. In 17th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’23). USENIX Association, Boston, MA, 701–718.
  34. [34]Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, et al. 2026. OpenAI GPT-5 System Card. arXiv:2601.03267 [cs.CL] https://arxiv.org/abs/2601.03267
  35. [35]Gemma Team. 2025. Gemma 3 Technical Report. arXiv:2503.19786 [cs.CL]. https://arxiv.org/abs/2503.19786
  36. [36]Qwen Team. 2026. Qwen3.5: Towards Native Multimodal Agents. https://qwen.ai/blog?id=qwen3.5.
  37. [37]Jianghui Wang, Vinay Joshi, Saptarshi Majumder, Xu Chao, Bin Ding, Ziqiong Liu, Pratik Prabhanjan Brahma, Dong Li, Zicheng Liu, and Emad Barsoum. 2025. Geak: Introducing Triton Kernel AI Agent & Evaluation Benchmarks. CoRR abs/2507.23194 (2025). https://doi.org/10.48550/ARXIV.2507.23194 arXiv:2507.23194
  38. [38]Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Daniel Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. 2025. OpenHands: An Open Platform for AI Software Developers as Generalist Agents. In International Conference on Learning Representations (ICLR ’25). 65882–65919.arxiv.org/abs/2407.16741
  39. [39]Zhaode Wang, Jingbang Yang, Xinyu Qian, Shiwen Xing, Xiaotang Jiang, Chengfei Lv, and Shengyu Zhang. 2024. MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices. In Proceedings of the 6th ACM International Conference on Multimedia in Asia Workshops (MMAAsia ’24 Workshops). Association for Computing Machinery, New York, NY, USA, Article 11, 7 pages.DOI
  40. [40]Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2025. Demystifying LLM-Based Software Engineering Agents. Proc. ACM Softw. Eng. 2, FSE (2025), 801–824. https://doi.org/10.1145/3715754
  41. [41]Chen Yang, Jiashuo Tian, Ziqi Wang, Xinyin Liu, Meiru Ye, and Junjie Chen. 2026. Learning Globally Reusable Skills for Coding Agents. CoRR abs/2608.06153 (2026). https://doi.org/10.48550/ARXIV.2608.06153 arXiv:2608.06153
  42. [42]John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-Agent: Agent-Computer Interfaces Enable Automated Software Engineering. In Neural Information Processing Systems (NIPS ’24).arxiv.org/abs/2405.15793
  43. [43]Songlin Yang, Jan Kautz, and Ali Hatamizadeh. 2025. Gated Delta Networks: Improving Mamba2 with Delta Rule. In The Thirteenth International Conference on Learning Representations (ICLR ’25).arxiv.org/abs/2412.06464
  44. [44]Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, et al. 2025. MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe. arXiv:2509.18154 [cs.LG] https://arxiv.org/abs/2509.18154
  45. [45]Xingyi Yuan, Morgane M. Moss, Charbel El Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, and Marc-Alexandre Côté. 2025. debug-gym: A Text-Based Environment for Interactive Debugging. CoRR abs/2503.21557 (2025). https://doi.org/10.48550/ARXIV.2503.21557 arXiv:2503.21557
  46. [46]Guibin Zhang, Junhao Wang, Junjie Chen, Wangchunshu Zhou, Kun Wang, and Shuicheng YAN. 2026. AgentTracer: Who Is Inducing Failure in the LLM Agentic Systems?. In The Fourteenth International Conference on Learning Representations (ICLR ’26).arxiv.org/abs/2509.03312
  47. [47]Kechi Zhang, Jia Li, Ge Li, Xianjie Shi, and Zhi Jin. 2024. CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL ’24). Association for Computational Linguistics, Bangkok, Thailand, 13643–13658.arxiv.org/abs/2401.07339
  48. [48]Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. AutoCodeRover: Autonomous Program Improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (Vienna, Austria) (ISSTA ’24). Association for Computing Machinery, New York, NY, USA, 1592–1604.arxiv.org/abs/2404.05427
  49. [49]Lianmin Zheng, Chengfan Jia, Mingxing Tan, Runzhao Wu, Cody Hao Yu, Ameer Haj-Ali, Yida Wang, Jun Yang, Danyang Zhuo, Koushik Sen, Joseph E. Gonzalez, and Ion Stoica. 2020. Ansor: Generating High-Performance Tensor Programs for Deep Learning. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’20). USENIX Association, 863–879.arxiv.org/abs/2006.06762
  50. [50]Hongyu Zhu, Ruofan Wu, Yijia Diao, Shanbin Ke, Haoyu Li, Chen Zhang, Jilong Xue, Lingxiao Ma, Yuqing Xia, Wei Cui, Fan Yang, Mao Yang, Lidong Zhou, Asaf Cidon, and Gennady Pekhimenko. 2022. ROLLER: Fast and Efficient Tensor Compilation for Deep Learning. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). USENIX Association, Carlsbad, CA, 233–248.
  51. [51]Kunlun Zhu, Zijia Liu, Bingxuan Li, Muxin Tian, Yingxuan Yang, Jiaxun Zhang, Pengrui Han, Qipeng Xie, Fuyang Cui, Weijia Zhang, Xiaoteng Ma, Xiaodong Yu, Gowtham Ramesh, Jialian Wu, Zicheng Liu, Pan Lu, James Zou, and Jiaxuan You. 2025. Where LLM Agents Fail and How They can Learn From Failures. arXiv:2509.25370 [cs.AI] https://arxiv.org/abs/2509.25370
  52. [52]Xingze Zou, Jing Wang, Yuhua Zheng, Xueyi Chen, Haolei Bai, Lingcheng Kong, Syed A. R. Abu-Bakar, Zhaode Wang, Chengfei Lv, Haoji Hu, and Huan Wang. 2026. MobileKernelBench: Can LLMs Write Efficient Kernels for Mobile Devices? CoRR abs/2603.11935 (2026). https://doi.org/10.48550/ARXIV.2603.11935 arXiv:2603.11935

Paper details

vlnc:W_6gksc88w
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 20

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents