betaFeedback

Reading v1View latest version

WorksComputer Science

NORmal: Architecting 3D Ferroelectric NOR Flash as Main Memory – From Co-Fabricated BEOL Arrays to 200B-Class Multi-Agent LLMs in a Smartphone

Abstract

Smartphone agents increasingly work as teams, in which a large planner directs parallel workers across long sessions. Running such a team on the device requires 200B-class models and contexts that exceed phone DRAM many times over, yet existing tiers treat flash as storage: UFS starves per-token weight streaming, and mobile High-Bandwidth Flash (HBF) needs a redesigned SoC and cannot absorb continuous KV writes. We show that a non-volatile memory can join an unmodified LPDDR6 bus as a plain rank only if its access latency fits the DRAM timing window, a condition no NAND tier meets. We present NORmal, to our knowledge the first main-memory architecture for 3D ferroelectric NOR flash. We have fabricated and experimentally characterized back-end-of-line (BEOL) 3D FeNOR arrays that co-integrate dense 3-bit MLC and fast, high-endurance in-place-writable SLC with sub-7.5 ns sensing. NORmal turns these co-fabricated arrays into a byte-addressable, deterministic-timing load/store rank on an unmodified LPDDR6 interface through an under-array DRAM-emulation bridge, staggered bank reads with zero-cycle weight swizzling, and posted SLC writes. Across this rank and DRAM, NORmal places agent state by mutability and lifetime rather than by hotness, serving 200B-class models and persistent planner contexts in place while dynamically partitioning subchannels across agent phases. In serving-level simulation on production agent traces, NORmal is 8.9×\times–28.0×\times faster than UFS offloading, within 0.4% of an unlimited-DRAM ideal, and 2.8×\times–6.3×\times faster than HBF on sessions beyond 256K tokens, with 8.1×\times–17.8×\times less memory energy per token.

Introduction

Smartphone agents now complete tasks rather than answer questions. They read screens and sensors, hear from you, operate apps, and act on messages and credentials, so they must run on the device for privacy. They also increasingly work as teams [3, 25]. A planner holds the long-running session and dispatches subtasks to parallel workers that run a smaller model, return a result, and exit. In the production traces we replay [34], a planner keeps up to ten workers in flight over sessions of up to 1,970 turns. Reliable planning needs large Mixture-of-Experts (MoE) models [8, 49], whose sparse activation keeps compute tractable on a mobile NPU. Memory is the bottleneck. A 200B-class planner alone exceeds usable DRAM by nearly 20×, since the operating system and background apps already hold over half of a phone’s memory [44].

Our characterization of these traces shows that the team’s memory demands follow its structure, and that each demand

NORmal MLC is written only at model installation; its write numbers refer to SLC.

HBF additionally requires a redesigned SoC, TSVs, and a custom SRAM base die at 3× UFS cost per byte. breaks the conventional split between DRAM and flash storage. (i) Both models must stay resident and be read at memory speed. DRAM holds only a few percent of a large MoE model, and because routers shift active experts across tokens, caching and prefetching still leave gigabytes of weights to fetch per token [38, 49]. Swapping a 167 GB planner over UFS instead costs 15 s whenever control passes between the planner and its workers. (ii) The planner’s context outgrows DRAM and is written at every token. Long multi-turn histories, which on phones also accumulate screenshots and audio [4, 19], push the planner’s KV well past usable DRAM. (iii) The team’s state splits by lifetime, and its traffic shifts by phase. Planner KV persists for the entire session, worker KV is discarded as soon as each worker returns, and bus demand alternates between planner decode and worker execution.

Existing mobile tiers cannot meet these demands because they treat flash as block storage rather than memory. UFS NAND provides ample capacity, but its narrow serial link and microsecond page reads throttle per-token weight streaming [2, 44]. Mobile High-Bandwidth Flash (HBF), such as Samsung’s zNAND-O [33, 40], widens the read link using 3D-stacked NAND, TSVs, a custom SRAM base die, a silicon interposer, and a redesigned SoC interface. However, HBF remains page-granular NAND. Coarse pages amplify fine-grained MoE fetches [22], and high write energy and limited endurance rule out per-token KV appends within a phone’s power and lifetime budgets. Once planner KV outgrows DRAM, HBF must spill it to UFS or recompute it.

Can flash instead serve as main memory on the bus the phone already has? An unmodified LPDDR6 controller assumes a fixed first-access latency and a fixed burst rate. Bank-level parallelism can supply the burst rate, but first-access latency is a device property that architecture cannot fix, and NAND misses it by more than 100×.

We have fabricated and experimentally characterized back-end-of-line (BEOL) 3D ferroelectric NOR (FeNOR) flash arrays that co-integrate dual-gate SLC and gate-all-around 3-bit MLC above CMOS logic (Figure 2). Our arrays combine three properties that turn flash into a plain rank on the existing LPDDR6 bus. (i) Measured sub-7.5 ns array sensing keeps a full read, including CMOS-under-Array (CuA) peripherals and the LPDDR6 PHY, inside the 25–35 ns DRAM timing window. (ii) A ≤400∘C\le400^{\circ}\mathrm{C} BEOL flow stacks 16–32 monolithic array layers directly above CMOS logic and sense amplifiers at 0.45 pJ/bit. (iii) One flow co-fabricates gate-all-around 3-bit MLC (with a measured 4.6 V eight-state memory window) for dense weights and dual-gate SLC (with >108>10^8 ON/OFF current ratio and >1011>10^{11} endurance) for 100 ns in-place writes on the same die. The tradeoff is explicit. FeNOR shares the bus’s 153.6 GB/s instead of adding a link as HBF does, and in return it provides writable capacity, sub-picojoule access, and unmodified SoC hardware. We present NORMAL, to our knowledge the first main-memory architecture for 3D ferroelectric NOR flash. Rather than acting as read-only execute-in-place ROM, NORMAL exposes co-fabricated MLC and SLC arrays as a byte-addressable, deterministic-timing load/store rank behind an unmodified LPDDR6 controller. In the CuA logic layer, staggered bank interleaving and a zero-cycle swizzle latch turn banks that sense at one third of the subchannel rate and store 3-bit cells into a continuous 4-bit weight stream at full bus speed. A posted-write buffer with pending-read forwarding hides 100 ns in-place SLC writes behind 15 ns DRAM write timing. Above this rank, NORMAL maps the agent hierarchy onto the memory hierarchy, placing data by mutability and lifetime rather than by hotness. Both models are immutable and always needed, so they live in MLC and are never swapped. The planner’s context is persistent and append-only, so it lives in in-place-writable SLC. Worker contexts are short-lived, so they stay in DRAM and are freed as each worker returns. Needless caching is unnecessary because, on a shared bus, a FeNOR read takes the same wire time as a DRAM read, so the NPU reads weights and planner KV in place. To follow the team’s phases, adaptive subchannel allocation re-divides the shared channel between FeNOR and DRAM at each phase boundary.

Fabricated 3D FeNOR arrays

Figure 2. Our fabricated 3D FeNOR arrays. (a) Schematics of the co-integrated dual-gate SLC (1 bit/cell) and gate-all-around MLC (3 bits/cell) arrays. (b, c) Top-view and cross-sectional SEM images at a 150 nm pitch. (d) Optical micrograph of the array test bank. (e, f) Cross-sectional TEM images of stacked SLC and MLC devices; adjacent channel layers are physically separated, which suppresses interlayer leakage. (g) SLC read current across the top-layer array. (h) MLC memory window across the top-layer array. Cells written to the same state show uniform responses.

We evaluate NORMAL with a full-stack serving simulator that replays single-agent and multi-agent sessions. NORMAl is 8.9×–28.0× faster than UFS 5.0 offloading (21.4× for the agent team) and reaches over 99.6% of the throughput of an unlimited-DRAM ideal. HBF’s dedicated link keeps it 1.10×–1.73× faster on light workloads, but without UFS spilling HBF cannot run up to 40% of sessions, and on sessions longer than 256K tokens NORMAL finishes 2.8×–6.3× faster. NORMAl also uses 8.1×–17.8× less memory energy per token than HBF, and SLC endurance removes write wear-out as a lifetime limit. We make four contributions:

  • Characterization of on-device agent teams (§ 2). On production traces and six models, we show that a planner–worker team needs resident models read at memory speed, writable capacity for planner KV beyond DRAM, and lifetime-aware placement, and that DRAM expert caching, UFS offloading, and HBF each fail at least one of these needs.

  • Fabricated 3D FeNOR as a plain LPDDR6 rank (§ 3). We separate a device-level latency bound from an architecture-level bandwidth bound for any memory behind an unmodified LPDDR6 controller, and show with fabricated and experimentally characterized BEOL 3D FeNOR arrays that co-integrated 3-bit MLC and in-place-writable SLC meet the latency bound.

  • The NORMAl architecture (§ 5). We present, to our knowledge, the first main-memory architecture for 3D ferroelectric NOR flash. A dual-tier MLC/SLC rank with a CuA DRAM-emulation bridge, zero-cycle weight swizzling, and posted SLC writes pairs with an

inference-in-place runtime to map the agent hierarchy onto the memory hierarchy.

  • Serving-level evaluation (§ 6). Against UFS 5.0 offloading, mobile HBF, and an unlimited-DRAM ideal, we report both where HBF’s extra bandwidth wins and where NORmal’s writable main-memory capacity wins.

Background and Characterization

Mobile Memories

A flagship smartphone SoC reaches DRAM through one 96-bit LPDDR6 interface organized as eight independent 12-bit subchannels, providing up to 153.6 GB/s [17]. A typical device ships with 16 GB of DRAM, but much less is available to any one application. Even on a freshly managed phone, PowerInfer-2 measures that a single app can occupy at most 11 GB of a 16 GB device [44]. In ordinary use, vendor Android builds keep many background apps and services resident. We therefore model an inference budget of 8 GiB on a 16 GB device, after the operating system, background services, and a few-MiB runtime reserve (Sec. 6.1). Table 1 compares the tiers that can extend this capacity.

UFS NAND. Phones store model weights in UFS 4.0/5.0 NAND flash, which provides hundreds of gigabytes behind a serial M-PHY link of 4.2–10 GB/s. Reads take tens of microseconds and fetch whole 4 KiB pages. Existing on-device systems stream the active parameters from UFS into DRAM and cache frequently used parameters in DRAM [2, 39, 44, 49].

Mobile HBF. High-Bandwidth Flash stacks NAND dies on a logic base die and connects them to the SoC through a dedicated wide-I/O link on a custom interposer [10, 12, 13, 18, 21, 22, 26, 27, 28, 35, 40, 45, 47, 50]. Samsung’s zNAND-O targets mobile devices [33]. It stacks four SLC NAND dies (4-Hi) for low read latency and integrates several MiB of SRAM in the base die to hide microsecond array latency. The link provides up to 200 GB/s. Passive cooling limits a stack to 4-Hi, which gives 64–128 GB per stack, so up to 171.2 GB of co-resident weights in our multi-agent workload needs 2–3 stacks. Reads cost 8 pJ/bit and writes cost 100 pJ/bit, endurance is 10510^{5} P/E cycles, and the TSVs, base die, organic substrate, and SoC redesign raise cost per byte to 3× that of UFS. Page-granular reads amplify fine-grained accesses, by up to 13.6× on 320 B rows [22].

On-Device Agent Workloads

Models. We study six models (Table 2): two dense models and four MoE models from 35B to 314B parameters. The MoE models activate 3B–17B parameters per token, and a 200 TOPS INT4 NPU has ample compute for single-user decode. The 100B-class MoE models have weight footprints of 94.3–166.5 GB at 4 bits, exceeding usable phone DRAM by up to 19×.

Mobile memory layouts

Figure 1. Mobile memory layouts: (a) off-chip UFS 5.0, (b) interposer Mobile HBF, and (c) NORmal on shared LPDDR6.

Agent structure. Personal agents decompose tasks into a planner–worker team. A planner running a large model holds the session, plans the task, and dispatches subtasks to workers that run a smaller, faster model in parallel. In the production agent traces we replay [34], the planner can have up to ten workers in flight at once. Our default team pairs a Qwen3.8-Flash-Next planner with Qwen3.5-9B workers. We choose same-family models so that the planner and its workers share a tokenizer and prompt format, which keeps context consistent as it passes between them and keeps the team comparable with the single-model Qwen workloads. Both models must stay resident. Reloading the Flash-Next planner over a 10.8 GB/s UFS link would take about 9 s.

Mobile agents add multimodal observations that increase memory pressure. A screenshot costs roughly 1,000–1,800 input tokens [4], and audio adds tokens in proportion to its duration [29, 42]. Production phone agents such as the Doubao phone assistant run tasks of 30 or more steps that switch between foreground and background execution, and mobile benchmarks already average 27.8 steps per task [19]. Keeping only a few recent observations would discard state these tasks need.

Characterization

Several works coordinate scheduling and KV-cache management using agent dependencies and tool-call intervals [5, 7]. We replay the traces against the six models with the simulator described in Sec. 6.1. Four findings follow, and each yields a requirement for the memory system.

Weight streaming and expert caching. Existing systems cache frequently used experts in DRAM and prefetch predicted ones [39, 44, 49]. Training balances expert load only globally, so individual requests and domains still favor subsets of experts [23, 30, 43]. Hidden-state similarity also lets systems predict next-layer experts with about 90% accuracy [37, 48]. Phone DRAM limits what this locality can buy. Even if all 8 GiB held experts, it would cover under 6% of a 151–167 GB model, and caches hit only 30–53% of accesses even at a 25% cache [38, 49]. Prediction hides fetch latency but not fetch volume, because every miss still crosses the UFS link. At 4 bits, the 3B–17B active parameters take 1.5–8.5 GB per token. Without cache hits, a 10.8 GB/s UFS link therefore caps decode at 1.3–7.2 tok/s, against 18–102 tok/s at full LPDDR6 bandwidth, and a cache with hit rate hh raises this cap only by a factor of 1/(1−h)1/(1-h). The cache also competes with KV for the same DRAM (Finding 2).

Finding 1

Phone DRAM is too small for expert caching to remove the UFS bottleneck.

Requirement: the capacity tier itself must deliver weights at memory bandwidth and latency.

KV growth. Using the per-token KV sizes in Table 2, a single Qwen3.8-27B session of the traces’ mean prompt length (223K tokens) needs 7.2 GiB of KV, about the entire usable DRAM. The longest session (962K tokens) needs 31 GiB. With one planner and four concurrent workers at 200K context, KV reaches 1.8×–2.4× the usable DRAM. This KV is appended at every decoded token and re-read at every attention step.

PlatformXiaomi XRING O3 Mobile SoC, 200 TOPS NPU, W4A8, INT8 KV
LPDDR696-bit (8 × 12-subch.), 153.6 GB/s
Host DRAM16 GB (8 GiB avail.)
NOR8-Hi×32L Cu 8A, 96 MB LC / 64 GB SLC, 4×/8× bank interleaving
ModeTier org.BW (DRAM/Flash)Cap. (DRAM/Flash)KV policy
UFS 5.0DRAM + Serial NAND153.6 / 10.8 GB/s8 GiB / 512 GBLRU, UFS swap
Mobile HBFDRAM + 3×4-Hi zNAND-O153.6 / 200 GB/s8 GiB / 192 GBLRU, UFS spill / recompute
NORmalShared LPDDR6 Rank 1Shared 153.6 GB/s8 GiB / 221 GBSLC/DRAM, Lifecycle-tiered
Ideal LPDDR6 DRAMUnlimited Rank 0153.6 GB/s / —∞ / 0 GBIn-DRAM full retention
TraceType (Sessions, Max Context)ReqsPˉ\bar{P}Gˉ\bar{G}
singleSingle-agent (215 sessions, ≤962K)17 122223 3971 159
longLong-context (62 sessions, ≥256K)10 276297 2661 309
multi-agent1 Planner + 4 Workers (164 sessions)54 91094 757815
ModelLayersExperts (Top-kk)Params (Act.)Model SizeKV / tok
Qwen3.5-9B32Dense (—)9B (8B)4.43 GiB17.00 KiB
Qwen3.8-27B64Dense (—)27B (26B)13.31 GiB34.00 KiB
Qwen3.6-35B-A3B40256 (Top-8)35B (3B)17.15 GiB10.63 KiB
Qwen3.8-Flash-Next48512 (Top-10)177B (7B)87.84 GiB13.15 KiB
DeepSeek-V4-Flash43256 (Top-6)285B (13B)140.69 GiB3.57 KiB
GLM-5.3-Flash45288 (Top-8)314B (17B)155.02 GiB6.21 KiB

Table 2. Evaluated Hardware, Agent Traces, and Models.

Finding 2

KV caches outgrow DRAM within a single session.

Requirement: the system needs a writable tier, beyond DRAM, with memory-speed reads and enough endurance for continuous appends.

KV lifecycles. The two kinds of KV behave differently. The planner’s KV is append-only and persists across the entire multi-turn session, so it grows large. Each worker’s KV lives only for one subtask and is discarded when the worker returns its result.

Finding 3

Planner KV persists for the whole session, while worker KV lives only for one subtask.

Requirement: persistent planner KV belongs in a writable capacity tier, while ephemeral worker KV belongs in DRAM so that it can be freed immediately.

Phase-dependent traffic. During planner decode, traffic is dominated by weight and planner KV reads. During worker execution and prefill, DRAM traffic (worker KV, activations, and recurrent state) grows. Any design that serves both kinds of traffic from one channel faces a demand mix that changes at phase boundaries; Sec. 6.4 shows that fixed splits lose 9%–87% of throughput.

Finding 4

The mix of capacity-tier and DRAM traffic shifts at agent phase boundaries.

Requirement: if capacity-tier and DRAM traffic share a channel, the split must follow the phase.

Limitations of Existing Memory Tiers

DRAM caching. DRAM provides the bandwidth, latency, and writability that Findings 1–4 require, but not the capacity. Up to 171 GB of co-resident weights is an order of magnitude beyond phone DRAM, and Finding 1 shows that caching a subset does not substitute for capacity.

UFS offloading fails the weight requirement of Finding 1, as its serial link caps decode at a few tokens per second. It also fails the KV requirement of Finding 2. Spilled KV must cross the same link, and the NAND endures only 10310^3–10410^4 P/E cycles.

Mobile HBF meets the weight requirement of Finding 1 with a dedicated link, although 4 KiB pages waste part of it on fine-grained reads. It fails the KV requirement of Finding 2 on a phone. Datacenter HBF systems buffer and coalesce append-only KV writes [40] under active cooling, but coalescing reduces write amplification without lowering the intrinsic 100 pJ/bit NAND programming energy. Under a phone’s passive thermal budget, sustained GB/s KV writes during long-context prefill or DRAM-overflow eviction would consume several watts in flash programming alone, forcing mobile HBF to operate as a read-only weight tier. Any HBF capacity left over after storing weights therefore sits idle. Even if thermal limits were ignored, 10510^5 P/E endurance would still wear out within months to a few years (Sec. 6.4). KV must stay in DRAM and be evicted once it exceeds DRAM (Finding 2). HBF also requires a redesigned SoC. The extra link is a real advantage while KV fits in DRAM.

No existing tier offers memory-speed, writable capacity on the interface the SoC already has. The next section shows when a memory can provide it.

Flash as a Plain LPDDR6 Rank

The Rank Condition

An unmodified LPDDR6 controller makes two timing assumptions about every rank [17]. After a row activate (ACT), the first column read returns within the programmed tRCD+RLt_{RCD}+RL. After that, 32 B bursts stream every two tCCD=1.67t_{CCD}=1.67 ns on each 12-bit subchannel (19.2 GB/s). A non-volatile memory can therefore serve read bursts as a standard rank behind an unmodified controller only if

taccess≤tRCD+RL,(1)t_{\mathrm{access}} \le t_{RCD}+RL, \tag*{(1)}
NbankUrowtbank≥32 BtCCD,(2)N_{\mathrm{bank}}\frac{U_{\mathrm{row}}}{t_{\mathrm{bank}}} \ge\frac{32\ \mathrm{B}}{t_{CCD}}, \tag*{(2)}

where UrowU_{\mathrm{row}} is the data one bank senses per cycle tbankt_{\mathrm{bank}}, and NbankN_{\mathrm{bank}} is the number of banks whose cycles can overlap. (1) is a device-level requirement that no amount of architecture can relax. (2) is an architecture-level requirement that adding banks can satisfy.

Prior non-volatile ranks. NAND flash misses (1) by more than 100× (4–50 μs), which is why UFS and HBF remain block storage behind their own interfaces. Lincoln [36] shares LPDDR channels between DRAM and NAND flash ranks to run 50–100B models on consumer devices. Because NAND cannot meet Eq. (1), Lincoln stages data in near-PHY SRAM using explicit commands and polling and computes generation-phase dot products near the flash, while KV stays in DRAM. Cambricon-LLM [46] similarly pairs an NPU chiplet with compute-enabled NAND and keeps KV in DRAM. Phase-change memory [20, 31] and Optane persistent memory, with ∼300 ns reads [15], also miss Eq. (1), and Optane used a non-deterministic DDR-T protocol with a modified controller. Ferroelectric 1T1C capacitor RAM (FeDRAM, such as Micron’s 2-layer, 32 Gb NVDRAM [32]) meets DRAM timing on LPDDR5, but it is a destructive-read 1T1C RAM rather than flash. Each read requires a write-back restore, cells store only 1 bit, and density stays at the DRAM scale (4 GB/die) rather than the 100 GB scale of multi-layer, multi-bit 3D flash. Sharing a mobile bus with NVM has precedent. JEDEC LPDDR2-NVM [16] let PCM and SDRAM share command and data pins with separate chip selects, and it defined NVM-specific initialization, buffering, and program operations. SHyLA [11] places weights and KV across 3D-stacked PCM and DRAM, but in a custom cloud accelerator rather than on a phone’s LPDDR bus.

NORmal WMCM package placing the 8-die 3D FeNOR stack beside the SoC and copper heat sink

Figure 3. NORmal WMCM package placing the 8-die 3D FeNOR stack beside the SoC and copper heat sink.

3D FeNOR Meets the Latency Bound

Conventional 3D NOR. NOR flash reads randomly in under 100 ns and is byte-addressable, so it can serve as execute-in-place memory rather than block storage; LPDDR4-interfaced NOR products already exist [14, 24]. Conventional floating-gate or charge-trap 3D NOR, however, lacks the multi-bit density to hold 150–200 GB of weights. It also needs front-end thermal processing above 800°C, which prevents stacking over CMOS logic, and its high-voltage charge pumps consume 22.5 pJ/bit when programming.

Our fabricated 3D FeNOR. We fabricate 3D ferroelectric NOR (FeNOR) arrays in a back-end-of-line (BEOL) flow at or below 400°C, using atomic-layer-deposited InGaZnO₄ (IGZO) channels and Hf₀.₅Zr₀.₅O₂ (HZO) ferroelectric gate dielectrics [9]. One flow co-integrates two cell structures (Figure 2a): dual-gate (DG) single-level cells (SLC, 1 bit per cell) and gate-all-around (GAA) multi-level cells (MLC, 3 bits per cell), both at a 150 nm pitch (Figure 2b–Figure 2f). The SLC array shows an ON/OFF read-current ratio above 10810^{8} (2 μA2\ \mu\mathrm{A} versus 10 fA10\ \mathrm{fA}; Figure 2g). The MLC array opens a 4.6 V memory window that separates eight threshold states (Figure 2h). Cells written to the same state respond uniformly across both arrays.

Properties for an LPDDR rank. Together with CMOS-under-Array (CuA) integration, our 3D FeNOR arrays provide three properties that satisfy the rank latency bound and Findings 1–3 (Table 1).

Memory TierCapacity per 96-bit InterfaceHost Interface & Peak Read BandwidthLatency ReadLatency WriteEnergy per Bit ReadEnergy per Bit WriteMinimal Access UnitEndurance (P/E Cycles)
LPDDR6 DRAM [17]12–24 GB96-b LPDDR6 (153.6 GB/s)30 ns15 ns1.5 pJ1.5 pJ32 B> 10^{15}
UFS 4.0/5.0 (3D NAND) [44]256–1024 GBSerial M-PHY (4.2–10 GB/s)40 μs500 μs>10 pJ>100 pJ4 KiB10^{3}–10^{4}
HBF (zNAND-O) [33, 40]64–128 GB (4-Hi)Custom Wide I/O + Substrate (200 GB/s)4 μs100 μs8 pJ100 pJ4 KiB10^{5}
NORmal (Ours)MLC: 49–197 GB SLC: 4–24 GBShared LPDDR6 (153.6 GB/s)30 ns100 ns (SLC)0.45 pJ0.45 pJ96 B / 64 B10^{8} / >10^{11}

Table 1. Mobile memory and flash tiers on one 96-bit interface.

  1. Nanosecond sensing (Eq. (1) and Finding 1). Measured cell and array characterization [1, 9] shows that both cell types sense in under 7.5 ns. Array sensing, CuA peripheral settling, and the LPDDR6 PHY total 25–35 ns, fitting within tRCD+RLt_{\mathrm{RCD}}+\mathrm{RL}. Unlike NAND flash, 1T1C FeDRAM, or conventional 1-bit NOR, 3D FeNOR combines multi-bit 3D flash density with sub-35 ns non-destructive reads inside the latency window of a plain LPDDR6 rank. A single bank, however, senses 96 B per 15 ns (6.4 GB/s), or one third of the 19.2 GB/s subchannel rate in Eq. (2); § 4.2 closes this gap with bank interleaving.

  1. Multi-layer BEOL density and sub-picjoule access (Finding 1). The ≤400∘C\le400^{\circ}\mathrm{C} BEOL thermal budget projects to 16–32 monolithic array layers per die at 85% CuA area efficiency without degrading underlying CMOS. Because sense amplifiers, write drivers, and the LPDDR6 bridge logic sit directly beneath the vertical bitlines, measured array sensing and calibrated SPICE/CACTI modeling of the CuA peripherals and PHY keep die-level read and SLC write energy at 0.45 pJ/bit, 18× below HBF read energy.

  1. Co-fabricated MLC and SLC modes on one die (Findings 2 and 3). The two cell structures trade write cost against density differently. GAA MLC stores 3× the bits per cell, programs rows via a 100 μ\mus erase-and-verify, endures 10810^{8} P/E cycles, and is more sensitive to thermal drift across its eight states, so it suits immutable weights. DG SLC switches a single cell in place in 100 ns without block erase, endures more than 101110^{11} P/E cycles, and is insensitive to drift, so it suits persistent KV appends.

LPDDR6 Support for a Flash Rank

NORmal relies on three standard LPDDR6 features [17]. First, mode registers (MR) set timing per rank, so Rank 1 (FeNOR) can advertise a 25–35 ns tRCD+RLt_{\mathrm{RCD}} + \mathrm{RL} and the deterministic 1.67 ns tCCDt_{\mathrm{CCD}} independently of Rank 0 (DRAM). Second, the eight independent 12-bit subchannels let DRAM and FeNOR traffic be divided per subchannel. Third, the 24-UI burst frame (BL24, 288 bits, carrying a 32 B payload and 32 metadata bits) on a 12-bit subchannel aligns with 12 bits, the least common multiple of 3-bit cells and 4-bit weights. In addition, on-the-fly 32 B/64 B burst control and dynamic non-target on-die termination (NT-ODT) support 96 B MLC weight fetches, 64 B SLC KV accesses, and fast rank turnaround.

Unmodified SoC hardware. The SoC’s memory controller, PHY, and NPU are unchanged. All NOR-specific logic (the DRAM-emulation bridge, swizzle latch, posted-write buffer, and scrubber) sits in the CuA layer beneath the array. The changes are software, relying on standard controller support for per-rank timing registers and extended row addressing. Boot firmware programs Rank 1’s mode registers and exposes the 221 GB rank via upper row-address bits that the CuA bridge decodes into die, layer, and bank selects. Because the controller already maps the eight 12-bit subchannels to distinct physical address slices with independent per-subchannel queues, the operating system maps the MLC and SLC regions as physical address ranges, and the NPU runtime steers bandwidth across subchannels purely in software by issuing DMA descriptors to the target subchannel address slices (Sec. 5.4).

Remaining Challenges

FeNOR physics and the LPDDR6 features make a memory-mapped flash rank feasible. Three gaps remain between the device and a working rank that meets the requirements of Findings 1–4 (Sec. 2.3).

Challenge 1: Read path (Finding 1). A single bank delivers one third of the subchannel rate (Eq. 2), and 3-bit MLC cells must deliver byte-aligned 4-bit weights. Padding each weight to two cells wastes 33% of capacity, and unpacking in software stalls the NPU vector pipeline.

Challenge 2: Write path and placement (Findings 2 and 3). A 100 μ\mus MLC erase-and-verify cannot absorb runtime KV appends. Even a 100 ns SLC write exceeds the DRAM write timing the controller expects (tWR≈15t_{\mathrm{WR}} \approx15 ns). Stacking 3-bit MLC directly on an 85°C SoC in package-on-package (PoP) accelerates drift across its eight states. Data placement must also follow the lifecycle split between persistent planner KV and ephemeral worker KV.

Challenge 3: Shared bus (Finding 4). Multi-gigabyte FeNOR streams of weights and planner KV share one 96-bit channel with latency-critical DRAM traffic (OS, activations, and worker KV). Static subchannel splits strand bandwidth as multi-agent execution moves between phases.

Overview

To an unmodified smartphone SoC, NORmal appears as a second LPDDR6 rank (Rank 1) sharing the 96-bit channel with LPDDR6 DRAM (Rank 0) via chip select. Its physical address space is partitioned into a read-only MLC aperture for co-resident model weights and a read/write SLC aperture for persistent planner KV caches, while Rank 0 DRAM hosts OS state, activations, and ephemeral worker KV caches. Figure 4 traces the three core runtime operations. ① Weight read: the NPU issues standard DMA reads to the MLC aperture; the under-array bridge converts each ACT into staggered bank activations, the swizzle latch aligns 3-bit cells into byte-aligned weights, and bursts stream directly into NPU SRAM without DRAM staging (Sec. 5.2). ② Planner KV append: planner KV lines are written via ordinary STOREs to the SLC aperture, absorbed at DRAM write timing by a posted-write buffer, drained into SLC banks in the background, and later re-read in place during attention (Sec. 5.3). ③ Adaptive bandwidth allocation: at each execution phase boundary, the runtime dynamically partitions the eight LPDDR6 subchannels between DRAM and FeNOR with work-conserving idle-channel borrowing (Sec. 5.4).

NORmal overview

Figure 4. NORmal overview. FeNOR joins DRAM as Rank 1 on an unmodified 96-bit LPDDR6 bus, placing data by lifetime: ① in-place MLC weight reads, ② posted SLC planner KV appends, and ③ phase-adaptive subchannel splits.

NORmal Architecture

Mutability- and Lifecycle-Tiered Organization

NORmal tiers data by write mutability and agent lifecycle rather than access hotness (Sec. 2.3). Because DRAM and FeNOR share the same LPDDR6 bus, reading from either rank consumes identical wire time, whereas copying “hot” experts into DRAM wastes both bus bandwidth and energy. Each FeNOR die therefore splits its 3D array into two co-fabricated regions above the CuA logic layer (Fig. 4). The **MLC region stores immutable weights—including both the large planner and the lightweight worker models—programmed once at model installation and read in place during inference, eliminating model swapping across planner and worker phases. The SLC region stores persistent, append-only planner KV caches, whose 100 ns single-cell writes and >1011>10^{11} endurance absorb long-horizon context growth without block erase or write amplification. Conversely, LPDDR6 DRAM holds mutable state (activations and OS memory) alongside ephemeral worker KV caches: because workers use a compact model and discard their context immediately upon completing a subtask, keeping their KV in DRAM avoids unnecessary flash writes and frees DRAM capacity as soon as each worker returns to the planner. Each NOR stack integrates H∈{4,6,8}H \in\{4, 6, 8\} vertically stacked 68.15 mm2^2 dies with L∈{16,32}L \in\{16, 32\} monolithic array layers per die over a single shared 96-bit LPDDR6 interface, scaling total stack capacity from 53.6 GB (H=4,L=16H = 4, L = 16) to 221 GB (H=8,L=32H = 8, L = 32), providing 49–197 GB of MLC weight capacity alongside 4–24 GB of SLC KV capacity. Across this design space, array depth LL sets per-die density within the sensing window while die count HH scales package capacity; partitioning the die area between MLC and SLC trades weight-capacity margin for larger on-package KV retention while comfortably fitting 100B–300B+ W4 models and co-resident worker models on an 8-die stack.

Thermal-aware packaging. Because 3-bit MLC threshold drift accelerates with temperature, placing the FeNOR stack atop an 85°C SoC in a conventional package-on-package (PoP) layout degrades state retention. NORmal instead adopts a wafer-level multi-chip module (WMCM) layout that places the FeNOR stack side by side with the SoC and DRAM under a shared vapor chamber (Fig. 3), keeping the stack near 50°C under sustained inference and slowing threshold drift by 3.5×. A lightweight CuA scrubber tracks thermal exposure and re-verifies drifted rows only while the phone is charging, so scrubbing never contends with inference.

Read Path: DRAM-Emulation and Swizzle Latch

Bus-aligned rows and DRAM-emulation bridge.

Each MLC bank senses 28=2562^8 = 256 cells per wordline (96 B, or three 32 B BL24 bursts), holding 192 4-bit weights (W4) or 256 3-bit weights (W3) with a power-of-two column decoder and no leftover bits; each SLC bank senses a 64 B KV line (two bursts). Supported by LPDDR6’s per-rank Mode Register (MR) timing configuration, the CuA bridge of each subchannel emulates deterministic DRAM read timing across the full stack capacity (addressed via upper row-address bits). Upon receiving ACT, the bridge triggers four MLC banks staggered by 5.0 ns to fill a rolling row buffer within the programmed tRCD+RLt_{\mathrm{RCD}} + \mathrm{RL} window (Eq. (1)), and subsequently streams 32 B bursts every tCCD=1.67t_{\mathrm{CCD}} = 1.67 ns on RD commands. Because FeNOR reads are non-destructive, PRE and REF commands update only bridge state. Four staggered banks provide 25.6 GB/s per subchannel—exceeding the 19.2 GB/s subchannel wire rate and providing slack against bank-cycle variations—to satisfy Eq. (2) (Figure 5(a)).

CuA datapath

Figure 5. CuA datapath: (a) staggered 4-bank MLC reads, (b) zero-cycle W4 swizzle latch, and (c) posted SLC writes.

Swizzle latch. Storing 4-bit weights in 3-bit MLC cells via naive two-cell padding wastes 33% of capacity, while software bit-unpacking stalls the NPU vector pipeline. The swizzle latch eliminates both overheads by exploiting the natural 12-bit alignment between LPDDR6 subchannels, four 3-bit cells, and three 4-bit weights. At model installation, every three 4-bit weights (12 bits) are striped across four 3-bit cells. During a read, the 256 sensed cells form 64 four-cell tuples; for each tuple mm, fixed metal routing between the sense amplifiers and the burst serializer directly wires the 12 sensed bits PmP_m into three consecutive 4-bit weight nibbles (Pm[11:8]P_m[11:8], Pm[7:4]P_m[7:4], Pm[3:0]P_m[3:0]) on the byte bus (Figure 5(b)), while a 2:1 mux supports native W3 mode. Because reassembly is pure wiring, the latch adds zero cycle latency and <0.02 mm2<0.02\ \mathrm{mm}^2 per die while increasing effective W4 capacity by 1.5× over padding. Quantization group scales are stored in a separate byte-aligned table (<1.6%<1.6\% metadata traffic).

Write Path: Posted SLC Writes

Although DG SLC switches cells in place in 100 ns without block erase, it remains slower than LPDDR6 write timing (tWR≈15t_{\mathrm{WR}} \approx15 ns). Each subchannel bridge therefore integrates a 4 KiB SRAM posted-write buffer (<0.08 mm2<0.08\ \mathrm{mm}^2 per die) that accepts 64 B planner KV lines at full bus speed and drains them in the background across eight interleaved SLC banks (Figure 5(c)). Together, eight banks per subchannel sustain 41 GB/s of aggregate write bandwidth across the 96-bit interface—more than 20× the peak KV append rate during inference (§ 5.4)—preventing buffer overflow. Read requests that hit pending lines in the buffer are served directly from SRAM, and the bridge schedules bank drains between attention read bursts (or steers writes to idle banks) so background SLC programming never blocks latency-critical reads.

Runtime and Adaptive Bandwidth Allocation

Lifecycle-aware placement. At model load time, the runtime maps each data class to its matching tier without modifying model code. Co-resident planner and worker model weights reside permanently in FeNOR MLC and stream directly into NPU SRAM via DMA. Planner KV caches are appended to FeNOR SLC and read in place during attention,

whereas ephemeral worker KV caches remain in LPDDR6 DRAM and are reclaimed immediately when a worker completes its subtask. When an extreme single-agent session on a KV-heavy dense model exceeds the default SLC aperture, which can be enlarged beyond the default by trading MLC area for SLC capacity, no workers are active in DRAM, so the runtime spills the remaining KV tail into the usable DRAM budget before ever touching UFS, absorbing the full session on package.

Adaptive bandwidth allocation. Because DRAM and FeNOR share the eight 12-bit LPDDR6 subchannels, static subchannel partitioning leaves bandwidth stranded as execution shifts between FeNOR-heavy planner weight/KV streaming and DRAM-heavy worker execution or prefill (Figure 6). Since weights and KV lines are striped across all eight subchannel address slices, the NPU runtime steers subchannel bandwidth in software without modifying the SoC memory controller. At each layer or agent-stage boundary, the runtime estimates the pending FeNOR traffic DNORD_{\mathrm{NOR}} (weights, routed experts, and SLC KV reads) and DRAM traffic DDRAMD_{\mathrm{DRAM}} (activations, worker KV, and reserved OS traffic), and dispatches concurrent FeNOR DMA descriptors to

LPDDR6 subchannel sharing

Figure 6. LPDDR6 subchannel sharing: (a) phase-adaptive split, (b) static 4:4 split, and (c) bounded DRAM preemption.

kNOR=clip⁡(round⁡(8DNORDNOR+DDRAM),1,7)(3)k_{\mathrm{NOR}} = \operatorname{clip}\left(\operatorname{round}\left(\frac{8D_{\mathrm{NOR}}}{D_{\mathrm{NOR}} + D_{\mathrm{DRAM}}}\right), 1, 7\right) \tag*{(3)}

active subchannel slices, rotating the active set across micro-batches so every address slice drains evenly while leaving at least one subchannel queue free of FeNOR DMA traffic for latency-critical DRAM requests. Crucially, the runtime pairs this phase-aware split with *work-conserving idle-subchannel borrowing: whenever either rank has no pending descriptors for its assigned slices within a scheduling window, the runtime steers ready DMA descriptors from the other rank into those idle subchannel queues rather than leaving bus wires unused. Finally, the runtime groups FeNOR DMA accesses on each active subchannel into contiguous batches of W=32W = 32 bursts (1 KiB); combined with the unmodified controller’s standard command batching and LPDDR6’s fast dynamic NT-ODT switching, this amortizes rank-switch turnaround while bounding high-priority DRAM preemption delay to at most W⋅tCCD=53.3W \cdot t_{\mathrm{CCD}} = 53.3 ns.

Evaluation

Methodology

Simulator and hardware configuration. We evaluate NORmal using a full-stack serving simulator built upon TokenSim [6, 41], modeling Xiaomi XRING O3, a flagship mobile SoC equipped with a 200 TOPS INT4/INT8 NPU and a 96-bit LPDDR6 memory subsystem. All configurations provision 16 GB of LPDDR6 DRAM, with 8 GiB pinned by the mobile OS and background applications. Of the remaining 8 GiB, about 0.5 GiB is reserved for the inference framework and runtime, leaving 7.46 GiB for weights, KV caches, and activations. FeNOR array timing and energy follow fabricated devices [1, 9] (Table 1) and assume a passive mobile thermal envelope. NORmal uses an 8-die, 32-layer FeNOR stack sharing the 96-bit LPDDR6 channel as Rank 1 (197 GB MLC for weights and up to 24 GB SLC for persistent KV caches).

Workloads and agent traces. Table 2 summarizes the seven evaluated workloads: two dense models, four sparse MoE models spanning 35B to 314B parameters, and a Multi-Agent planner–worker team that pairs a Qwen3.8-Flash-Next planner with four concurrent Qwen3.5-9B workers; we use same-family models so that planner and workers share a tokenizer and prompt format (§ 1.2). To capture

realistic long-horizon agent execution, we replay 215 intact single-agent coding sessions up to 962K context tokens and 164 multi-agent sessions with up to 1,970 turns and four-way parallel worker delegation from production coding-agent traces [34].3 Weights use group-quantized W4 (packed via the swizzle latch; model sizes in Table 2 include group-quantization scales and non-expert layers) and KV caches use INT8 with architecture-native recurrent state formats. Baselines. We compare three mobile memory systems alongside an idealized reference: (1) UFS 5.0 [44], which pairs LPDDR6 DRAM with 10.8 GB/s UFS 5.0 flash storage and demand-fetches uncached weights and spilled KV blocks; (2) Mobile HBF (zNAND-O) [33, 40], which integrates a 3-stack mobile HBF tier providing a dedicated 200 GB/s read-mostly weight link (10510^5 P/E endurance, 4 KiB page access) while keeping active KV caches in LPDDR6; (3) NORmal (Ours), which attaches the dual-tier MLC/SLC FeNOR stack directly to the shared 96-bit LPDDR6 bus with adaptive sub-channel allocation; and (4) Ideal LPDDR6, an upper-bound reference with unlimited LPDDR6 DRAM capacity.

End-to-End Throughput, Latency, and Energy

End-to-end throughput and energy. Figure 7(a) compares pooled end-to-end throughput across all seven workloads. Because UFS 5.0 is bottlenecked by its 10.8 GB/s serial link and tens-of-microseconds page latency during weight streaming and KV swapping, it sustains only 0.29–4.05 tok/s. By executing weights in place from MLC and retaining long-horizon KV caches in SLC at LPDDR6 bus speed, NORmal achieves 6.50–37.26 tok/s, delivering an 8.9×–28.0× end-to-end speedup over UFS 5.0 (26.1× on Flash Next, 28.0× on GLM 5.3, and 21.4× on Multi-Agent). Against Mobile HBF, the bandwidth tradeoff of § 2.2 is visible. On six of the seven workloads, HBF’s dedicated 200 GB/s link gives it 1.10×–1.73× higher throughput, and NORmal reaches 58%–91% of HBF. Decode on these sessions is bound by weight streaming, and HBF adds a link while NORmal shares the LPDDR6 bus. On Qwen 35B, NORmal outperforms Mobile HBF (35.9 vs. 32.9 tok/s) even on HBF’s supported session subset, because HBF repeatedly recomputes prefixes when inactive session caches are evicted from DRAM. § 5.3 shows that the ranking reverses once KV caches exceed DRAM.

End-to-end throughput and memory energy across seven workloads

Figure 7. End-to-end performance across seven workloads: (a) pooled output throughput (tok/s, excluding trace idle time) and (b) memory subsystem energy per generated token (J/tok).

Figure 7(b) shows that NORmal reduces memory subsystem energy to 0.016–0.068 J/tok, an 11.3×–27.0× reduction over UFS 5.0 and an 8.1×–17.8× reduction over Mobile HBF (0.13–0.92 J/tok). Mobile HBF’s high energy stems from activating wide 4 KiB NAND pages and waking multi-TSV HBF dies, whereas NORmal senses only exact 96 B/64 B lines over low-capacitance CuA routing.

Prefill TTFT and decode TPOT. Figure 8 breaks down latency into mean prefill TTFT and decode TPOT under an identical KV retention policy across systems. During prefill

Latency breakdown under identical KV retention

Figure 8. Latency breakdown under identical KV retention: (a) mean prefill TTFT and (b) mean decode TPOT.

(Figure 8(a)), UFS 5.0 incurs 3.3–29.0 s of TTFT because cold model weights and historical KV prefixes must stream across the storage link. NORmal cuts mean TTFT to 0.9–3.3 s (2.1×–21.6× faster than UFS 5.0). On the KV-intensive dense model (Qwen 9B: 1.6 s vs. 1.7 s), NORmal has slightly lower TTFT than Mobile HBF, which must spill or rebuild evicted prefixes. HBF is 0.1–0.2 s faster on the three other workloads. During decode (Figure 8(b)), NORmal reduces TPOT from 455–792 ms on UFS 5.0 to 26–54 ms (8.4×–26.1× lower). It cuts Qwen 9B TPOT by 1.75× relative to Mobile HBF (54 ms vs. 95 ms). On the MoE and multi-agent workloads, HBF’s extra link gives it 1.1×–1.3× lower TPOT.

Task Finish Time, Long-Horizon Scaling, and Multi-Agent Gain

Session-level task finish time. Without UFS KV spilling, a read-only weight tier fails once a session’s KV footprint exceeds usable DRAM: Mobile HBF fails capacity checks on 87 of 215 sessions (40.5%) for Qwen 27B and 20 of 215 sessions (9.3%) for Qwen 9B. With UFS spilling and prefix recomputation enabled so every session in the cohort runs to completion, Figure 9 reports normalized task finish time relative to the fastest system (Base). Across the full cohort (Figure 9(a)), NORmal is the fastest system (Base) on both Qwen 27B (where Mobile HBF takes 4.81× longer and UFS 5.0 takes 23.54× longer) and Qwen 35B (where HBF takes 1.15×1.15\times longer), while remaining within 1.10×1.10\times–1.19×1.19\times of HBF’s dedicated 200 GB/s link on GLM 5.3 and Multi-Agent. On the 62 long-horizon tasks exceeding 256K context tokens (256K–962K, Figure 9(b)), NORmal is Base across all four workloads: Mobile HBF takes 2.78×2.78\times–6.33×6.33\times longer (6.33×6.33\times on Qwen 27B, 3.21×3.21\times on Qwen 35B, 3.71×3.71\times on GLM 5.3, and 2.78×2.78\times on Multi-Agent), and UFS 5.0 takes 8.20×8.20\times–31.31×31.31\times longer.

Normalized task finish time

Figure 9. Normalized task finish time (Base = 1.00×): (a) all sessions and (b) long-horizon sessions (>256K tokens).

Why NORmal beats HBF on long horizons. Figure 10(a) contrasts NORmal’s speedup over Mobile HBF between the pooled cohort and >256>256K/peak sessions. Once active and retained KV caches exceed the 8 GiB DRAM budget, Mobile HBF cannot absorb KV writes on package within a phone’s passive thermal and endurance budgets; it must either spill KV to UFS 5.0 (incurring 47.6 TB of cumulative UFS reads on Qwen 27B) or evict and recompute up to 499M prefix tokens on GLM 5.3. NORmal absorbs up to 23.6 GB of persistent KV caches in byte-addressable FeNOR SLC (plus up to 8 GiB in usable DRAM on extreme single-agent sessions) and reads SLC KV in place without promotion to DRAM, reaching up to 11.98×11.98\times single-session speedup on Qwen 27B and 8.08×8.08\times on Qwen 35B.

Long-horizon and multi-agent speedups

Figure 10. (a) NORmal speedup over Mobile HBF on pooled vs. >256K sessions, and (b) team speedup over Planner-Only.

Multi-agent cooperation. Figure 10(b) evaluates a team of one planner and four workers. Beyond the default Flash Next planner, it also varies the planner (Qwen 27B, DS-V4 Flash, or GLM 5.3) while keeping the Qwen 9B worker model in FeNOR MLC. Because both models remain permanently resident in MLC while the four concurrent workers allocate their ephemeral KV caches in LPDDR6 DRAM, NORmal eliminates the tens of gigabytes of UFS model-swapping traffic required by conventional mobile runtimes and achieves a 1.05×1.05\times–1.36×1.36\times pooled speedup and 1.18×1.18\times–1.92×1.92\times peak session speedup over running all turns on the planner model (Planner-Only).

Adaptive Bandwidth Allocation and Device Endurance

Adaptive bus sharing vs. static splits. Figure 11 compares NORmal’s adaptive subchannel allocator against static DRAM-to-FeNOR subchannel reservations (Strict 7:1, 1:1, and 1:7 without idle borrowing), normalized to Ideal LPDDR6 (Base). Static 7:1 starves FeNOR weight streaming and loses 81%–87% of throughput, while static 1:1 loses 39%–48%. Even a FeNOR-heavy Strict 1:7 split loses 9%–32% of throughput (losing 29% on Qwen 9B and 32% on Qwen 35B) because a single DRAM subchannel throttles DRAM-resident recurrent state and KV traffic. By dynamically matching sub-channel shares to phase demand (Eq. 3) with work-conserving idle borrowing, NORmal incurs only 0.12%–0.31% throughput loss (>99.6%>99.6\% of Ideal LPDDR6) across all six models. Furthermore, on the read path, four-bank interleaving with the zero-cycle swizzle latch eliminates the 5.4×5.4\times–11.3×11.3\times 4 KiB page read amplification of NAND tiers and expands effective W4 weight capacity by 1.5×1.5\times over two-cell padding, enabling the 155.02 GiB GLM 5.3 model (Table 2) to fit within a single 8-die stack.

Throughput under static subchannel splits versus NORmal adaptive allocation

Figure 11. Throughput under static subchannel splits versus NORmal adaptive allocation, normalized to Ideal LPDDR6.

Write volume and multi-year device endurance. Finally, Figure 12 evaluates flash write endurance on the longest multi-turn coding session (962K context, 794 requests, 812K generated tokens; and 1,970 requests for Multi-Agent). Because a phone’s passive thermal budget forbids sustained multi-watt 100 pJ/bit NAND programming during normal operation, we evaluate a relaxed upper bound for Mobile HBF that ignores thermal limits, permits KV offloading into HBF, and wear-levels writes across the entire post-weight slack capacity of the 192 GB HBF package (177.7 GB on Qwen 27B, 97.7 GB on Flash Next, 40.9 GB on DS-V4 Flash, and 20.8 GB on Multi-Agent). Even with buffered append-only writes [40], 4 KiB page granularity and repeated KV eviction and promotion between DRAM and flash write 3.5–112.2 TB to flash in a single session, consuming 110–632 P/E cycles per session across the slack capacity (Figure 12(a)). Under one such heavy agent session per day, Mobile HBF (10510^5 P/E limit) still wears out in just 0.43–2.48 years (5.2 months on Qwen 27B and 7.0 months on Flash Next), falling well short of the 5-year smartphone lifespan requirement even under this generous wear-leveling assumption (Figure 12(b)). In contrast, because NORmal writes only newly appended 64 B KV lines in place to its 23.6 GB SLC region without block erase and reads historical SLC KV blocks directly in place without promoting them back to DRAM, it writes only 18.6–170.3 GB per session (0.80.8–7.27.2 P/E cycles, a 30×30 \times–659×659 \times reduction in physical write volume). Combined with FeNOR’s >1011> 10^{11} P/E endurance, NORmal comfortably exceeds the 5-year smartphone target (>107> 10^{7} years), eliminating write wear-out as a deployment barrier for on-device agents.

Endurance on the 962K-context session

Figure 12. Endurance on the 962K-context session: (a) flash P/E cycles per session and (b) projected lifetime in years.

Conclusion

We presented NORmal, the first main-memory architecture for 3D ferroelectric NOR flash. It turns flash from block storage into byte-addressable inference memory by attaching our fabricated dual-tier 3D FeNOR as a native second rank on an unmodified mobile LPDDR6 interface. Our characterization shows that on-device large-model agents need memory-speed weight access from a capacity tier, writable capacity for KV beyond DRAM, and lifecycle-aware KV placement. Because FeNOR meets the latency bound of an LPDDR rank, a deterministic DRAM-emulation bridge with four-bank interleaving, a zero-cycle W4/W3 swizzle latch, a lifecycle-tiered MLC/SLC die with posted SLC writes, and an adaptive bus-sharing runtime are enough to host 200B-class models and long-horizon agent KV caches on package.

References

  1. [1]Xuez heng Ai, Weiwei Li, Chen Gu, Chuanke Chen, Xinlv Duan, Congyan Lu, Jianqi Chen, Xiangsheng Wang, Kaiping Zhang, Jin Dai, Mingxu Liu, Jiebin Niu, Chuanhui Huang, Jinjian Xiang, Yong Yu, Feng Shao, Guan hua Yang, Yu Liu, Xiaomeng Liu, Shaohua Wang, Bok-Moon Kang, Gengfei Li, Shenjie Zhao, Nianduan Lu, Di Geng, Guilei Wang, Chao Zhao, Ling Li, and Ming Liu. 2024. First Demonstration of 4-layer Stacked Planar Channel-All-Around (P-CAA) IGZO FETs with Cost-effective Process for High-density 1T1C 3D DRAM. In Proc. IEEE Int. Electron Devices Meeting (IEDM). doi:10.1109/IEDM50854.2024.10873307DOI
  2. [2]Keivan Alizadeh, Iman Mirzadeh, Dmitry Belenko, Karen Khatamifard, Minsk Cho, Carlo C. Del Mundo, Mohammad Rastegari, and Mehrdad Farajtabar. 2024. LLM in a Flash: Efficient Large Language Model Inference with Limited Memory. In Proc. Annu. Meeting Assoc. Comput. Linguistics (ACL).
  3. [3]Anthropic. 2026. Claude Code: Subagents. https://code.claude.com/docs/en/sub-agents. Accessed 2026-10-28.
  4. [4]Anthropic. 2026. Computer Use Tool: Screenshot History and Token Consumption. https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool. Accessed 2026-10-28.
  5. [5]Zhuohang Bian, Feiyang Wu, Zhuoran Li, Teng Ma, and Youwei Zhuo. 2026. TokenCake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applications. arXiv:2510.18586 [cs.DC] https://arxiv.org/abs/2510.18586
  6. [6]Zhuohang Bian, Feiyang Wu, Zhuoran Li, Yibo Zhao, Junchi Wu, Xu-xiao Yang, and Youwei Zhuo. 2026. TokenSimV2: Accurate and Fast LLM Inference on GPUs via Dynamic Graph Modeling. In International Symposium on Advanced Parallel Processing Technologies (APPT). Springer Nature Singapore, Singapore.DOI
  7. [7]Zhuohang Bian, Feiyang Wu, Chengrui Zhang, Hangcheng Dong, Yun Liang, and Youwei Zhuo. 2026. TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing. arXiv:2604.03143 [cs.DC] https://arxiv.org/abs/2604.03143
  8. [8]DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. arXiv preprint arXiv:2412.19437 (2024).arxiv.org/abs/2412.19437
  9. [9]Chen Gu, Haochen Yu, Zhaori Cong, Weiwei Li, Qianqian Wei, Zhenkun Chen, Yuan Xue, Junzhe Shen, Minggao Zhan, Yiyang Yuan, Chuanke Chen, Cailian Ma, Wenfeng Jiang, Xinlv Duan, Xin Liu, Zuowei Zhang, Congyan Lu, Shengjie Zhao, Kaiping Zhang, Jiebin Niu, Yu Liu, Pengfei Jiang, Jinshan Yue, Guanhua Yang, Zhenchong Wu, Nianduan Lu, Di Geng, Qing Luo, Ling Li, and Ming Liu. 2026. Hybrid Analog–Digital LLM-CiM Enabled by 3D Co-Fabricated Vertical Dual-Gate SLC and GAA MLC FeFET Arrays with Distinct Architectures. In Proc. IEEE Int. Electron Devices Meeting (IEDM).
  10. [10]Minho Ha, Euis eok Kim, and Hoshik Kim. 2026. H3: Hybrid Architecture Using High Bandwidth Memory and High Bandwidth Flash for Cost-Efficient LLM Inference. IEEE Computer Architecture Letters 25, 1 (2026), 49–52. doi:10.1109/LCA.2026.3660969
  11. [11]Liu He, Fuyao Zhou, Cheng Peng, Shun Dong, Ziming Zhang, Huazhong Yang, Yongpan Liu, Guowei Zhang, and Hongyang Jia. 2026. ShyLa: 3D-Stacked NVM-DRAM Hybrid LLM-Inference Architecture Exploiting Data and Memory Heterogeneity. In Proc. Int. Symp. Computer Architecture (ISCA).DOI
  12. [12]Po-Kai Hsu, Weihong Xu, Qunyiou Liu, Tajana Rosing, and Shiming Yu. 2026. HAVEN: High-Bandwidth Flash Augmented Vector Engine for Large-Scale Approximate Nearest-Neighbor Search Acceleration. arXiv:2603.01175 [cs.AR] https://arxiv.org/abs/2603.01175
  13. [13]Yangshu Hu, Yuwei Yang, Yuan Wu, Kexin Chu, Yusheng Zheng, Andi Quinn, and Wei Zhang. 2026. HBFSim: Fast and Faithful Simulation of High-Bandwidth Flash Under Real GPU Execution. arXiv:2609.09800v1 [cs.AR] https://arxiv.org/abs/2609.09800v1
  14. [14]Infineon Technologies. 2023. SEMPER X1 LPDDR4-Interfaced NOR Flash Memory Architecture. Product Specification.
  15. [15]Joseph Izraelevitz, Jian Yang, Lu Zhang, Juno Kim, Xiao Liu, Amirsaman Memaripour, Yun Joon Soh, Zixuan Wang, Yi Xu, Subramanya R. Dulloor, Jishen Zhao, and Steven Swanson. 2019. Basic Performance Measurements of the Intel Optane DC Persistent Memory Module. arXiv preprint arXiv:1903.05714 (2019).arxiv.org/abs/1903.05714
  16. [16]JEDEC Solid State Technology Association. 2010. JESD209-2B: Low Power Double Data Rate 2 (LPDDR2). Technical Report. JEDEC. Includes LPDDR2-NVM.
  17. [17]JEDEC Solid State Technology Association. 2025. JESD209-6: LPDDR6 Standard Specification. Technical Report. JEDEC.
  18. [18]Joungho Kim. 2026. HBF Workload Analysis and Roadmap. Presentation, TeraByte Interconnect and Package Laboratory, KAIST, February 10, 2026.
  19. [19]Quyu Kong, Xu Zhang, Zhenyu Yang, Nolan Gao, Chen Liu, Panrong Tong, Chenglin Cai, Hanzhang Zhou, Jianan Zhang, Liangyu Chen, Zhidan Liu, Steven Hoi, and Yue Wang. 2025. MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments. (2025). arXiv:2512.19432 [cs.CL] https://arxiv.org/abs/2512.19432
  20. [20]Benjamin C. Lee, Engin Ipek, Onur Mutlu, and Doug Burger. 2009. Architecting Phase Change Memory as a Scalable DRAM Alternative. In Proc. Int. Symp. Computer Architecture (ISCA).
  21. [21]Jinheng Li, Guosheng Wang, Hongchao Du, Shangyu Wu, Qiao Li, and Chun Jason Xue. 2026. HBFSim: A Simulator for Exploring HBF in Memory Hierarchies for LLMs. arXiv preprint arXiv:2610.23503 (2026).DOI
  22. [22]Zhuoran Li, Zhuohang Bian, Xin Huang, Yibo Zhao, Guangyu Sun, and Youwei Zhuo. 2026. HBF Sucks? A Full-Stack Characterization of High-Bandwidth Flash for KV-Centric LLM Serving. arXiv preprint arXiv:2608.11668 (2026).arxiv.org/abs/2608.11668
  23. [23]Jingcong Liang, Siyuan Wang, Miren Tian, Yitong Li, Duyu Tang, and Zhongyu Wei. 2026. Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models. In Proc. Int. Conf. Learning Representations (ICLR).arxiv.org/abs/2505.16056
  24. [24]Hang-Ting Lue, Teng-Hao Yeh, Chih-Chang Hsieh, Tzu-Hsuan Hsu, ChihWei Hu, Chia-Jung Chiu, Guan-Ru Lee, Ken-Hui Chen, Chin-Hung Chang, Keh-Chung Wang, and Chih-Yuan Lu. 2026. A 4Gb 32-Layer 3D NOR Flash Test Chip with Innovative Design Methods Enabling Sub-100ns Low-Latency Read and Robust Retention. In 2026 IEEE/JSAP Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits). 1–3. doi:10.1109/VLSITechnologyandCir65830.2026.11575554DOI
  25. [25]Moonshot AI. 2026. How to Choose: K2.6 / K3 / K3 Cluster, Reasoning Effort, and Agent. https://www.kimi.com/en/help/others/model-mode-selection. Accessed 2026-10-28.
  26. [26]Geraldo F. Oliveira, Arash Tavakkol, Xiangyu Zhu, Ahmet Caner Yüzügüler, Vamanan Arulchelvam, Lukas Cavigelli, Renzo Andri, Mohammad Sarasdatı, Xinglei Jia, Onur Mutlu, Ke Zhou, Shai Bergman, and Ji Zhang. 2026. FLINT: Efficient Leveraging High Bandwidth Flash for Capacity-Scalable LLM Inference Acceleration. arXiv preprint arXiv:2608.25062 (2026).arxiv.org/abs/2608.25062
  27. [27]Open Compute Project (OCP). 2025. High-Bandwidth Flash (HBF) Technical Specification. Technical Report. Open Compute Project.
  28. [28]Vinicius Petrucci, Felip​​pe Zacarias, and Vishal Tanna. 2026. Is High-Bandwidth Flash All You Need? In 3rd Workshop on Hot Topics in System Infrastructure (HotInfra), co-located with ISCA.
  29. [29]Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shih​​ao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, Wanjun Zhong, Kuan​​ye Li, Jiale Yang, Yu Miao, Woyu Lin, Longxiang Liu, Xu Jiang, Qianli Ma, Jingyu Li, Xiaojun Xiao, Kai Cai, Chuang Li, Yaowei Zheng, Chaolin Jin, Chen Li, Xiao Zhou, Minchao Wang, Haoli Chen, Zhaojian Li, Haihua Yang, Haifeng Liu, Feng Lin, Tao Peng, Xin Liu, and Guang Shi. 2025. UT-PARS: Pioneering Automated GUI Interaction with Native Agents. arXiv preprint arXiv:2501.12326 (2025).
  30. [30]Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiy​​ue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025. Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models. In Proc. Annu. Meeting Assoc. Comput. Linguistics (ACL). doi:10.18653/v1/2025.acl-long.249DOI
  31. [31]Moinuddin K. Qureshi, Vijayalakshmi Srinivasan, and Jude A. Rivers. 2009. Scalable High Performance Main Memory System Using Phase-Change Memory Technology. In Proc. Int. Symp. Computer Architecture (ISCA).
  32. [32]N. Ramaswamy, A. Calderoni, J. Zahurak, G. Servalli, A. Chavan, S. Chh​​ajed, M. Balakrishnan, M. Fischer, M. Hollander, D. P. Ettisserry, A. Liao, K. Karda, M. Jerry, M. Mariani, A. Visconti, B. R. Cook, B. D. Cook, D. Mills, A. Torsi, C. Mouli, E. Byers, M. Helm, S. Pawlowski, S. Shiratake, and N. Chandrasekaran. 2023. NVDRAM: A 32Gb Dual Layer 3D Stacked Non-volatile Ferroelectric Memory with Near-DRAM Performance for Demanding AI Workloads. In Proc. IEEE Int. Electron Devices Meeting (IEDM). doi:10.1109/IEDM45741.2023.10413848
  33. [33]Samsung Electronics. 2025. Z-NAND:0-4-Hi 3D Stacked Low-Latency NAND Flash Architecture for On-Device and Mobile AI. Future of Memory and Storage (FMS) Keynote.
  34. [34]SemiAnalysis and Weka. 2026. Claude Code Multi-Turn Agent Traces Dataset. Hugging Face Datasets (https://huggingface.co/datasets/semianalysis/cc-traces-weka-062126).
  35. [35]Dowon Son, Yonggon Park, Hyunkyuk Cho, Hyungkyu Ham, Onur Mutlu, Sungjin Lee, Gwangsun Kim, and Jisung Park. 2026. Exploring High-Bandwidth Flash for Modern LLM Inference: Opportunities and Challenges. IEEE Computer Architecture Letters (2026). doi:10.1109/LCA.2026.3705817DOI
  36. [36]Weiyi Sun, Mingyu Gao, Zhaoshi Li, Aoyang Zhang, Iris Ying Chou, Jianfeng Zhu, Shaojun Wei, and Leibo L​​iu. 2025. Lincoln: Real-Time 50~100B LLM Inference on Consumer Devices with LPDDR-Interfaced, Compute-Enabled Flash Memory. In Proc. IEEE Int. Symp. High-Performance Computer Architecture (HPCA). 1734–1748.DOI
  37. [37]Peng Tang, Jiacheng Liu, Xiaofeng Hou, Yifei Pu, Jing Wang, Peng-An Heng, Chao Li, and Minyi Guo. 2024. HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference. arXiv preprint arXiv:2411.01433 (2024).arxiv.org/abs/2411.01433
  38. [38]tcclaviger. 2026. Qwen3.8-Flash-Next Expert Activation Map Dataset. Hugging Face Datasets (https://huggingface.co/datasets/tcclaviger/Qwen3.8-Flash-Next-expert-activation-map).
  39. [39]Haodong Wang, Qihua Zhou, Zicong Hong, and Song Guo. 2025. D²MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving. In Proc. ACM Int. Conf. Mobile Comput. Netw (MobiCom).
  40. [40]Xinyu Wang, Yalong Xue, Xiaotian Sun, Xiaoyu Zhang, Xinjiang Zhang, Chun​​meng Dou, Xueqi Li, and Xiaoming Chen. 2026. FlashAccel: Leveraging High-Bandwidth Flash (HBF) for High-Throughput LLM Inference. arXiv preprint arXiv:2607.10816 (2026).
  41. [41]Feiyang Wu, Zhuohang Bian, Guoyang Duan, Tianle Xu, Junchi Wu, Teng Ma, Yongqiang Yao, Ruihao Gong, and Youwei Zhuo. 2025. Token-Sim: Enabling Hardware and Software Exploration for Large Language Model Inference Systems. In International Symposium on Advanced Parallel Processing Technologies (APPT). Springer Nature Singapore, Singapore.arxiv.org/abs/2503.08415
  42. [42]Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. 2024. OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. In Advances in Neural Information Processing Systems (NeurIPS).arxiv.org/abs/2404.07972
  43. [43]Leyang Xue, Yao Fu, Zhan Lu, Luo Mai, and Mahesh Marina. 2024. MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache. arXiv preprint arXiv:2401.14361 (2024).arxiv.org/abs/2401.14361
  44. [44]Zhenliang Xue, Yixin Song, Zeyu Mi, Le Chen, Yubin Xia, and Haibo Chen. 2024. PowerInfer-2: Fast Large Language Model Inference on a Smartphone. arXiv preprint arXiv:2406.06282 (2024).arxiv.org/abs/2406.06282
  45. [45]Yihan Yin, Guangying Wu, Shu Li, Yilun Zhao, Feng Zhu, Fei Huang, and Shuangchen Li. 2026. Thoughtful Applications of HBF in LLM Serving Systems. arXiv:2608.13127v2 [cs.AR] https://arxiv.org/abs/2608.13127v2
  46. [46]Zhongkai Yu, Shengwen Liang, Tianyun Ma, Yunke Cai, Ziyuan Nan, Di Huang, Xinkai Song, Yifan Hao, Jie Zhang, Tian Zhi, Yongwei Zhao, Zidong Du, Xing Hu, Qi Guo, and Tianshi Chen. 2024. Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLM. In Proc. IEEE/ACM Int. Symp. Microarchitecture (MICRO). doi:10.1109/MICRO61859.2024.00108
  47. [47]Zack Yu, Chloe Wong, Coleman Hooper, Minjae Lee, Won​​jun Kang, Youngjin Cho, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. 2026. Characterizing High Bandwidth Flash for LLM Serving. arXiv:2609.39131v1. arXiv:2609.39131 Version 1, September 30, 2026.arxiv.org/abs/2609.39131
  48. [48]Shuzhang Zhong, Ling Liang, Yuan Wang, Runsheng Wang, Ru Huang, and Meng Li. 2024. AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE Inference. In Proc. IEEE/ACM Int. Conf. Computer-Aided Design (ICCAD). doi:10.1145/3676536.3676741DOI
  49. [49]Shuzhang Zhong, Y​​anfan Sun, Ling Liang, Runsheng Wang, Ru Huang, and Meng Li. 2025. HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference. In Proc. ACM/IEEE Design Automation Conf. (DAC).arxiv.org/abs/2504.05897
  50. [50]Shuzhang Zhong, Weikai Xu, Yifan Zhou, Tongbin Zhao, Tenghao Zhao, Yifei Kang, Cunyi Chang, Shu Li, Guangyu Sun, and Meng Li. 2026. HBFlex: A Flexible Memory System for Bridging Fine-Grained HBM States and Coarse-Grained HBF Parallel Execution. arXiv:2609.18675v1 [cs.AR] https://arxiv.org/abs/2609.18675v1

Paper details

vlnc:W_km7npctd
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 2

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents