WorksComputer Science
Chiron: Cache Management Empowered by LLM Intelligence for AI Workloads
Abstract
AI clusters keep data in remote object stores and rely on local caches to stop GPUs from waiting for data. A cache performs well only when its prefetching and eviction policies match the access pattern, yet these patterns differ across AI workloads and change between phases of one workflow. Existing methods either need code changes, which compromises user transparency, or use low-end heuristics, which cannot follow such changes. Large language models (LLMs) can infer access behavior without user hints, but direct LLM control of cache actions suffers from long reasoning delay, high token cost, and poor decision quality. We present Chiron, a cache system based on advisory switching control. An LLM asynchronously diagnoses the access pattern and recommends a policy. A deterministic harness gates this advice, executes expert-designed policies locally, and returns cache feedback for review. We implement Chiron atop JuiceFS and evaluate it on nine AI workloads. It reduces job completion time by 14.3-41.2% over the best baseline on popular AI workloads.
Introduction
AI clusters run diverse workloads, from data preprocessing to model training and evaluation [74] (He et al. 2016; Hu et al. 2022; Li et al. 2021). As datasets and models grow, these clusters keep data in remote object stores such as Amazon S3 [6] and serve it through a local cache layer such as Alluxio or JuiceFS [3, 43]. Remote accesses can stall input pipelines and leave GPUs waiting for data [61, 63, 46], while local capacity is limited. Prefetching and eviction policies therefore crucially affect the cache hit ratio (CHR) and job completion time (JCT).
AI workloads make these policies hard to set. Sequential, random, and skewed accesses are all common, each prefers different policies, and one workflow may pass through all three as it moves between phases (§ 2.2). An AI cache thus needs both adaptivity, matching its policy to the current pattern, and transparency, doing so without user hints (§ 2.3). Existing methods do not provide both (§ 3.1). Code-intrusive adaptations such as Quiver [47] and NoPFS [24] exploit workload knowledge through customized interfaces, which compromises transparency. Low-end adaptations such as ARC [58], LeCaR [83], and SFP [84] need no hints, but they follow variations within one pattern rather than transitions between patterns, which compromises efficacy.
Providing both requires inferring the upper-level computation logic from low-level accesses, a task that fits large language models (LLMs), which already assist networking (Wu et al., 2024) and database [95] management. Directly controlling the cache with an LLM, however, is impractical. We build a prototype in which an LLM lists the blocks to prefetch or evict after every read, and it reveals three challenges (§ 4):
Long reasoning delay. Qwen3.8 Flash takes 41 s at P50 even with thinking off, while a storage client can issue hundreds of reads per second.
High token cost. 128 reads consume over 10 M tokens, so a workflow of about 60k reads would need about 5 billion.
Poor decision quality. The model chooses among up to 1,963 candidate blocks and may hallucinate; the best prefetching accuracy across models is 48.2% lower than that of SFP.
We present Chiron, a fast, cheap, and effective cache system for AI workloads based on advisory switching control. As in supervisory switching control [62, 56], the LLM recognizes the operating regime and selects among expert-designed policies instead of acting on individual blocks. Since an LLM may hallucinate, it only advises: a deterministic harness decides whether to adopt its advice and returns cache feedback (§ 5). This design addresses the three challenges as follows:
Asynchronous diagnosis. Inference stays off the read path: reads continue under the active policy while the LLM reasons in the background. This suffices because access patterns persist across many reads.
Pattern-level advice. The LLM returns a short pattern label with a confidence score rather than block lists, and it runs only for routine checks or when cache feedback requests a review. One diagnosis then serves many reads, which keeps token cost low.
Harnessed execution. Choosing among three patterns replaces choosing among thousands of blocks. Deterministic executors pick objects locally, a confidence gate defers uncertain advice, and feedback-triggered review revises outdated diagnoses, which contains hallucination.
We implement Chiron atop JuiceFS with about 4,000 LoC of Go and 400 LoC of Python, and evaluate it on nine AI workloads with data on remote storage (§ 8). On three multi-phase workflows, Chiron achieves the highest CHR and reduces JCT by 14.3–41.2% over the best baseline. Ablations confirm that structured statistics, confidence gating, and feedback reflection are each necessary. In a simulation with a 1 TB dataset, Chiron also has the highest CHR and lowest JCT at every tested cache size.
Background
Caches Used for AI Workloads
AI techniques have substantially transformed many fields such as computer vision ((He et al., 2016)), natural language processing ([82]), and autonomous driving ([12]). Developing them requires diverse workloads such as data preprocessing, training, and inference, which are hosted in GPU clusters. With the drastic increase of dataset and model sizes, many enterprises such as Tencent ([94]) and RedNote ([4]) now store data remotely in cloud object stores such as AWS S3 ([6]) and Azure Blob ([60]). Such a compute-storage disaggregated architecture (Figure 1) allows storage capacity to scale independently of GPU resources, attaining low cost and high reliability.
Figure 1. Caching across AI workloads and workflow phases.
Under compute-storage disaggregation, caches are commonly adopted to improve remote I/O performance. Accessing remote storage incurs network latency and consumes bandwidth, which can stall input pipelines and leave GPUs waiting for data ([61, 63, 46]); to mitigate such overhead, frameworks such as Alluxio and JuiceFS provide a caching layer between computation and remote storage ([3, 43]). As shown in Figure 1, workflows access data through a POSIX-facing cache framework that retains data locally and serves cache hits without remote I/O.
Local cache capacity is limited relative to the datasets it serves, so appropriate cache-management policies are essential. Two key aspects are prefetching, which decides what data to fetch before it is requested ([24, 27, 84]), and eviction, which decides what resident data to remove when space is needed ([68, 90, 92]). These policies crucially affect the cache hit ratio and further the end-to-end execution time of workloads.
Unique I/O Properties of AI Workloads
Effective caching depends on how workloads traverse data, and AI workloads differ fundamentally from conventional ones ([58, 25] (Gu et al. 2023)) in two aspects.
(1) Diverse temporal access patterns across AI workloads. AI clusters support a wide range of workloads (Figure 1), including model training ([19, 45] (Qwen Team 2026)), model evaluation ([91, 22] (Li et al. 2026)), and data preprocessing ([67, 30]). These tasks differ in how they traverse data (Table 1). There are in general three typical patterns: sequential, for workloads like preprocessing and testing; random, for workloads like training ([61]); and skewed (i.e., with hotspots), for workloads like quality analysis and RAG ([40]).
| Temporal patterns | Workloads | Example access sequence | Desired prefetching policy | Desired eviction policy |
| Sequential | LLM model loading ((Fu et al., 2024)), LLM tokenization ([71]), · · · | 1.jpeg → 2.jpeg → 3.jpeg → 4.jpeg | Stride prefetching ([16]) | Eager eviction ((Li et al., 2008)) |
| Random | Vision fine-tuning ([23]), Speech fine-tuning ([75]), · · · | 10969.jpeg → 13625.jpeg → 19003.jpeg → 8077.jpeg | No prefetching ([8]) | No eviction ([61]) |
| Skewed | Recommendation training ([65]), RAG retrieval ([40]), · · · | A.csv → B.csv → A.csv → A.csv | Neighborhood prefetching ((Li et al., 2004)) | LRU eviction ([57]) |
Table 1. Representative file access patterns with example workloads, access sequences, and desired cache policies.
(2) Dynamic temporal access patterns within one AI workflow. An AI development workflow may comprise multiple dependent tasks, each a phase such as preprocessing, training, or evaluation (Figure 1). For example, NVIDIA NeMo connects data curation, model training, evaluation, and deployment in one workflow ([78]), and data-generation workflows alternate generation, filtering, de-duplication, and example selection ([21]). The access pattern over one dataset may thus change at any moment and is hard to identify in one shot.
In summary, AI workloads exhibit high diversity and dynamicity in temporal I/O characteristics, involving three comparably common access patterns. By contrast, conventional workloads such as operating systems ([20]), databases ([96]), and big data analytics ([7]) usually have one dominant pattern, typically skewed. This distinctness yields distinct design requirements for AI caches.
Design Requirements of AI Caches
Caching preferences follow I/O patterns. Different access patterns desire different prefetching and eviction policies (Table 1). Sequential accesses prefer stride prefetching (i.e., prefetching sequential items) and eager eviction (i.e., removing an item once it is accessed). Random accesses prefer no prefetching, as a low success rate wastes bandwidth and can pollute the cache ([8]), and no eviction, as eviction has no impact when each sample is accessed once per epoch ([61]). Skewed accesses prefer neighborhood prefetching (e.g., based on data-path locality) and LRU eviction, the classical policy adopted in caching frameworks like Alluxio. A mismatched policy lowers the cache hit ratio and prolongs job completion (§ 8).
Cache management requirements: adaptivity and transparency. An AI developer usually maintains a dedicated cache ([42] (Alluxio 2026c)), so the policy should match each workload at each instant. This adaptivity must not come at the price of user transparency: application hints through customized cache APIs bring a code-intrusion burden that is impractical for public users ([47]). It is thus desirable for the cache to select suitable policies automatically, without user hints. In a nutshell, we need an easy-to-use (with preserved user transparency), high-efficacy (with good workload adaptivity) cache for modern AI workloads.
Motivation
Limitations of Existing Works
Adaptivity and transparency are both desirable for an AI cache, yet they conflict, and existing works do not achieve
both.
Code-intrusive adaptations compromise user transparency. Given the distinct policy preferences of different access patterns (Table 1), some existing works adapt through customized caching interfaces. For example, Quiver exploits sample substitutability in shuffled training ([47]), and NoPFS predicts training accesses from the shuffle seed ([24]). Extending them to other phases requires new knowledge of how those phases access data and which cache behavior suits them, which conflicts with using the cache without workload-specific modifications or manual tuning.
Low-end adaptations compromise caching efficacy. Given the limitations of interface-based adaptations, other methods adopt simple heuristics or lightweight learning that automatically adjust cache policies according to the observed access history. For eviction, ARC balances recency and frequency ([58]), and LeCAR uses regret minimization to choose between LRU and LFU ([83]); for prefetching, SFP analyzes access correlations between files with a Markov chain and adapts its prefetching window ([84]). Without workload hints, however, these methods handle microscopic variations within one pattern (e.g., skewed), not macroscopic transitions across the patterns in Table 1. As a result, their cache hit ratio is substantially below that of code-intrusive adaptations (Figure 2).
Figure 2. Caching performance (cache hit ratio) of low-end and code-intrusive adaptations on ResNet-50 workflow (W1 in Table 2). For code-intrusive, we manually select the most suitable prefetching or eviction policy for each workflow phase according to its access pattern.
Lessons learned: the need for LLM intelligence to attain high-end adaptivity. Given the limitations of both code-intrusive and low-end adaptations, what we need is a sufficiently flexible caching method that works smoothly for arbitrary AI workloads. Under the transparency requirement, the cache itself must infer the upper-level computation logic, and then the desired policies in Table 1, from low-level access sequences. Given heterogeneous data semantics, intra-workflow dynamics, and the difficulty of telling random from skewed patterns, deterministic heuristics can hardly do so. Large language models (LLMs) ([13]) have the potential to fill this gap: with their zero-shot problem-solving capability, they have been exploited in networking ((Wu et al., 2024)) and databases ([95]).
| System prompt | Input | Output |
| Minimize future cache misses using recent block accesses and resident cache state. Return block IDs to prefetch and evict. | "accesses": [..., 7428, 2280, 347, 30692]<br>"resident": [[id, bytes], ...] | "prefetch": [id_1, ...]<br>"evict": [id_2, ...] |
Table 3. Input/output interface of our end-to-end cache-control prototype.
We therefore explore how to exploit LLM intelligence in cache management for AI workloads.
Challenges in Enforcing LLM-Based Adaptations
While promising, exploiting LLM intelligence in adaptive cache management is non-trivial. Here we empirically elaborate the challenges therein. As shown in Figure 3, we build a straightforward prototype atop JuiceFS that exploits LLM intelligence in an end-to-end manner: given a JSON input with recent block accesses and the resident cache state, the model outputs the block IDs to prefetch or evict after every read request. We measure1 its behavior with Qwen3.8 Flash on a small-scale diffusion LoRA development workflow (§ 8.1), a 1,024-access history, and a 1 GiB cache. The results reveal three challenges.
Challenge-I: long reasoning delay. Inference is far too slow for real-time control on the read path. Cloud-storage clients can issue hundreds of read requests per second: Bodner et al. report 250–350 asynchronous read requests per second from a single client instance accessing S3 (Bodner et al., [11]. Synchronous cache control makes read requests wait for model decisions, so inference must keep pace with frequent data accesses. Yet Qwen3.8 Flash takes 136 s at P50 and 210 s at P95 with thinking on, and 41 s at P50 with thinking off and capped output (Figure 4, left).
Figure 4. Qwen3.8 Flash overhead over 128 calls per configuration. Left: latency CDF. Right: token cost including input and output tokens. Both include the two cases of thinking-on and thinking-off.
Challenge-II: high token cost. The prototype repeatedly submits the updated history and resident state (Figure 3). Over 128 read requests, total tokens reach 11.84 M with thinking on and 10.61 M with thinking off, so the full diffusion LoRA workflow (about 60k read requests) would require about 5 billion tokens.
Challenge-III: poor decision quality. Despite their reasoning capability, LLMs make less accurate cache decisions than existing heuristics (Figure 5). The prefetching accuracy of the best LLM is 48.2% lower than that of SFP, and even the highest model CHRs stay below the baselines: 80.8% (DeepSeek) versus 82.3% (SFP) for prefetching, and 82.6% (Kimi) versus 83.9% (LeCaR) for eviction. A key reason is the large decision space: the LLM must choose prefetching targets from 1,963 candidates and eviction targets from 63. Potential LLM hallucination further lowers decision quality.
Figure 5. Cache-action quality. Prefetching accuracy is the fraction of prefetched blocks used before eviction or the end of the workload. Eviction accuracy is the fraction of evicted blocks next requested beyond the Belady boundary (Song et al., [77]) or not requested again in the remaining trace.
In summary, to bring LLM-powered adaptive caching into practice, we need to shorten the reasoning delay, reduce the token cost, and also improve the decision quality.
Solution
Overview
Design insight: advisory switching control. We propose Chiron, a fast, cheap, and effective cache system for AI workloads. Instead of end-to-end LLM control, Chiron borrows supervisory switching control from control theory [62, 56]: a supervisor recognizes the operating regime and switches among pre-designed controllers, each encapsulating expert knowledge for one regime (Figure 6a). Replacing continuous tuning with discrete switching responds quickly to large regime changes and keeps the system modular and controllable, balancing LLM flexibility with reliability and efficiency. Unlike a trusted supervisor, an LLM may hallucinate. We thus adopt advisory switching control (Figure 6b): the LLM only recommends a policy, while a deterministic harness decides whether to adopt it and returns cache feedback so the LLM can revise its advice.
Figure 6. Supervisory versus advisory switching control. (a) A trusted supervisor switches () among controllers based on output . (b) In Chiron, the LLM only advises; a deterministic harness gates its advice, switches among policy tools, and returns feedback for review. Thick arrows carry switching authority; dashed arrows are asynchronous.
Architecture and workflow. As Figure 7 shows, Chiron has two layers. In the LLM intelligence layer, an executor-switching advisor asynchronously diagnoses the access pattern: a runtime prompt packages recent FUSE accesses, precomputed statistics, and review context, and the advisor combines it with a fixed system prompt and returns a short JSON with a pattern and its confidence. In the LLM harness layer, the statistical analyzer prepares the statistics; the policy executor orchestrates one policy tool per pattern and adopts a recommendation only if it passes a confidence gate, after which the active executor makes every prefetching, admission, and eviction decision locally; and the feedback drafter monitors cache outcomes and requests routine or early checks with a drafted context.
Figure 7. Chiron architecture.
Why fast, cheap, and effective. Chiron is fast because inference stays off the read path and outputs are short; this asynchrony suffices because access patterns persist across many reads. It is cheap because the LLM selects policies instead of listing blocks and is invoked only when routine checks or feedback call for review. It is effective because choosing among expert-designed policies shrinks the decision space from thousands of candidate blocks to three patterns, while deterministic execution, gated adoption, and feedback-triggered review contain hallucination.
LLM Intelligence Layer
A key part of the Chiron design is the input and output format of the LLM advisor, summarized in Figure 8.
Figure 8. The diagnosis interface combines fixed instructions and three groups of runtime input with a compact pattern-level response. Only key instructions and selected runtime prompt fields are shown; ellipses omit details given in Appendix A. The response is illustrative.
Succinct yet Informative LLM Input
The input must support accurate judgment at low token cost. We therefore combine expert knowledge in a system prompt with runtime cache status in a runtime prompt. The system prompt describes the diagnosis task, which encodes the pattern knowledge of Table 1, explains how to interpret the runtime prompt, and constrains the output format (§ 5.3). The runtime prompt has three parts, numbered 1–3 in Figure 8; Appendix § 11 gives both prompts in full.
Access sequence. Diagnosis depends on access order and recurrence. Access sequence preserves the observed sequence so the advisor can inspect traversal and repeated objects directly. The harness copies a recent window2 from each cache unit, a file or directory managed together, keeping submitted evidence fixed while reads continue.
Structured statistics. Deriving stride and reuse statistics from raw IDs is inefficient and can produce unreliable judgments.
Structured statisticstherefore supplies precomputed values from the statistical analyzer, such asaccess_entropy, which provide cues for sequential, random, and skewed patterns at two window scales (§ 6.1).
| ID | Workloads | Dataset | Dataset Size (Cache Budget) | Access Pattern |
| W1 | ResNet-50 workflow (He et al., 2016) | ImageNet subset (ILSVRC) | 1.395 GiB (1 GiB) | Sequential, random, skewed |
| W2 | Minimal-FNO workflow [81] | PDEBench (PDEBench) | 7.67 GiB (4 GiB) | Sequential, random, skewed |
| W3 | Stable Diffusion 1.5 workflow (Stable Diffusion) | Mercity LAION subset (Mercity) | 3.87 GiB (1 GiB) | Random, sequential |
| T1 | Qwen tokenization task (Qwen Team) | C4 shards (Allen Institute for AI) | 2.37 GiB (2 GiB) | Sequential |
| T2 | Qwen2.5-3B weight loading task (Qwen Team) | Qwen2.5-3B weights (Qwen Team) | 5.75 GiB (4 GiB) | Sequential |
| T3 | ViT-Base fine-tuning task [23] | COCO 2014 subset (NaiveDev) | 6.09 GiB (1 GiB) | Random |
| T4 | Wav2Vec2 fine-tuning task [9] | Speech Commands v0.02 (Google) | 2.48 GiB (1 GiB) | Random |
| T5 | DLRM training task [65] | Criteo x1 trace (RECZOO) | 1.99 GiB (1 GiB) | Skewed |
| T6 | RAG retrieval task [50] | Wikipedia / MMLU (Cyro1; Center for AI Safety) | 3.04 GiB (0.5 GiB) | Skewed |
Table 2. Workloads named by model in the end-to-end experiment. IDs label the workloads in Figure 11 and Figure 12.
Review context. An accepted diagnosis may be incorrect or become outdated as accesses change. The feedback drafter therefore requests a new diagnosis when cache feedback suggests a mismatch (§ 6.3.1). To interpret this request, the advisor needs the earlier decision and the reason for reconsidering it, which
Review contextsupplies.
Token-efficient LLM Output
Listing actions for individual objects would lengthen the response as the working set grows. We instead return the compact pattern-level diagnosis in Figure 8, which limits generation work and lets the policy executor reuse an accepted diagnosis and choose objects locally. The pattern (p) specifies which prefetching and eviction strategies to consider for adoption (§ 6.2). Since the advisor may hallucinate a pattern unsupported by the evidence, the system prompt also asks for a self-assessed confidence score (c) in [0, 1], which the confidence gate uses (§ 6.3). An optional brief reason (r) can be enabled during debugging.
LLM Harness Layer
The LLM harness layer prepares statistical references, executes timely actions, defers unsupported policy changes, and reconsiders diagnoses as accesses change, through the four components below.
Statistical Analyzer for Reliable Evidence
The statistical analyzer, essentially a tool for LLM use, computes the structured statistics introduced in § 5.2.1 from the access sequence and supplies them to the runtime prompt. These statistics serve as references for assessing how closely the sequence matches the three access patterns. Figure 8 shows three example metrics, which provide clues for sequential, random, and skewed access, respectively. For a sequence , dominant_steps counts each adjacent ID difference and reports up to five most frequent differences with their counts, exposing repeated traversal steps. To compute access_entropy, the analyzer counts occurrences of each distinct ID and normalizes them as . With distinct IDs, the metric is , with for ; larger values indicate more evenly distributed access frequencies. Finally, top_5_access sums the counts of the five most frequent IDs and divides by : , where is the th largest count. Appendix A.2 gives the complete set of metrics. The analyzer also reports these statistics separately for the full window and its recent tail. This helps the advisor identify recent behavior during pattern transitions, when accesses from the previous phase may dilute the evidence in the full-window summary.
Deterministic Policy Executor for Fast Action
Cache contents can change during inference, making object choices from a diagnosis snapshot outdated. The deterministic policy executor therefore applies the active policy to current accesses and cache state, selecting prefetching and eviction actions locally. The sequential executor uses stride prefetching; the random executor disables pattern-based prefetching and rejects new admissions when full to retain the cached subset; the skewed executor combines neighborhood prefetching with LRU eviction. An accepted pattern change switches the active executor. Reads initially use the framework’s default policy, and pending diagnoses leave the active policy unchanged.
Confidence Gate for Hallucination Avoidance
To control inference cost, some deployments may use models with limited reasoning capability as advisors. Such advisors may hallucinate an access pattern, leading the executor to adopt an unsuitable cache policy. We observe that hallucinated diagnoses often coincide with low model confidence, so the score (§ 5.3) can identify some of them before adoption. For example, in the training phase of the ResNet-50 workflow with randomly shuffled accesses (§ 8.1), filtering diagnoses with while retaining the existing policy improves CHR from 13.84% to 15.45% (Figure 9). The confidence gate therefore requires before adopting a proposed pattern , and otherwise retains the existing policy. Since a simple filter does not correct a wrong diagnosis, the harness collects more accesses and retries sooner than the routine interval, allowing fresh evidence to help correct the diagnosis.
Figure 9. Patterns and confidence scores of diagnoses from Qwen3-Max (released in September 2025 and relatively dated compared with models we evaluate in § 8) (left) and CHR with and without a confidence filter (right) during the training phase of the ResNet-50 workflow.
Feedback Reflection for Fewer LLM Calls
Invoking the LLM periodically creates a tradeoff: frequent calls consume tokens even when the last accepted diagnosis still matches the access pattern, while longer intervals can delay adaptation when the pattern changes. Figure 10 illustrates this tradeoff in a replay of the ResNet-50 workflow, using a quarter of the dataset and cache budget. The feedback drafter therefore monitors runtime I/O with lightweight local rules and requests an early LLM review when necessary, allowing less frequent routine reviews.
Figure 10. Diagnosis schedule tradeoff in the ResNet-50 workflow (W1 in Table 2): JCT and token cost under different diagnosis intervals.
The drafter uses CHR fluctuations as a sign that the current pattern may no longer fit. For example, a random-to-skewed transition can raise CHR through increased reuse, yet the random executor’s fixed cached subset can exclude new hot objects that the skewed executor would admit. Let be CHR in round , and its exponentially weighted moving average (EWMA), with smoothing weight . A review is requested when , where is the fluctuation threshold. The drafter then sends the CHR fluctuation and the previous diagnosis to the advisor in the Review context (§ 5.2.1), alerting it that the earlier diagnosis may be outdated.
Implementation
We implement Chiron as an extension to JuiceFS, comprising a client-side controller written in Go and an LLM diagnosis server written in Python, with about 4,000 LoC of Go and 400 LoC of Python. The controller implements the policy executor and the feedback drafter of the LLM harness layer; the server implements the statistical analyzer and hosts the executor-switching advisor.
Diagnosis service and asynchronous interaction. The server connects the advisor to the LLM through LangChain’s chat-model interface and exposes it to the controller via a FastAPI endpoint. To keep cache control asynchronous with LLM inference, the controller snapshots the access history and review context, then launches a background goroutine for the HTTP exchange while foreground reads continue under the active policy. The server constructs the runtime prompt from this input and from statistics computed on the access history, and returns the diagnosis as JSON.
Client-side controller. To obtain the access sequence in Figure 7, we add a hook to JuiceFS’s FUSE Read() handler that collects each demand request’s inode, offset, and length. The controller converts this information into block IDs using the file-to-block mapping in metadata and records them in access order. The feedback drafter uses a background goroutine to periodically compute CHR fluctuations and update the review context. The policy executor calls VFS.Read() to fetch prefetch targets and removes selected cache files through the chunk store, which calls os.Remove(). Prefetch reads bypass the foreground hook to keep speculative accesses out of the access history.
Scalability. To maintain effective cache management and controlled diagnosis cost as datasets, caches, and access histories grow, Chiron keeps LLM input and output lengths bounded. The advisor uses a recent window of access history and returns a compact pattern-level diagnosis, so asynchronous inference can overlap with foreground reads as scale increases. Bounded inputs and outputs also control the token cost of each diagnosis, and the feedback drafter reduces the frequency of reviews.
Evaluation
We evaluate Chiron from five perspectives. First, we measure its end-to-end effectiveness across multi-phase and single-phase workloads (§ 8.1). Second, we assess the advantage of its pattern-aware eviction and prefetching over fixed policies (§ 8.2). Third, we test the necessity of structured statistics, confidence gating, and feedback reflection (§ 8.3). Fourth, we analyze how LLM capability affects Chiron (§ 8.4). Finally, we assess its cache performance at large dataset scale in simulation (§ 8.5).
End-to-End Performance
Workload and hardware setup. We use nine representative AI workloads (Table 2): three multi-phase workflows, which involve both pattern heterogeneity and dynamicity, and six single-phase tasks, which involve only pattern heterogeneity. Computations run on a server with eight A800 GPUs and an SSD-based cache, and datasets are stored remotely on S3 at a measured network latency of roughly 250 ms. To avoid trivial cases where the entire dataset can be cached without prefetching or eviction, we set a modest cache size for each experiment based on the original dataset size. The dataset and cache sizes3 are listed in Table 2.
Baselines. We compare Chiron with four baselines: (1) No-cache, which fetches demand data from remote storage without retaining read blocks; (2) JuiceFS, which uses sequential block prefetching and 2-random eviction, repeatedly evicting the older of two sampled blocks (JuiceFS); (3) Alluxio4, using its default LRU eviction without prefetching; and (4) JuiceFS-Adaptive, a variant of JuiceFS using LeCaR for eviction and SFP for prefetching. Later micro-benchmarks use other prefetching and eviction baselines.
Chiron setup. By default, the LLM is DeepSeek-V4-flash; § 8.4 tests other models. We set the confidence threshold to 0.70 (§ 6.3), and for feedback reflection (§ 6.3.1) the EWMA smoothing weight to 0.35 and the absolute CHR fluctuation threshold to 0.18. Appendix § 12 analyzes the sensitivity to this threshold.
Metrics. We report block-level cache hit ratio (CHR) and end-to-end job completion time (JCT), which are common in the literature ([44, 47]). Micro-benchmarks also report token cost and pattern diagnosis accuracy.
Overall improvements. As Figure 11 shows, adopting a cache substantially reduces JCT. More importantly, Chiron has the highest CHR and lowest JCT on multi-phase workflows, reducing JCT by 41.2%, 18.8%, and 14.3% over the best baseline on the three workflows. This confirms the power of LLM intelligence in adjusting the caching policy at runtime. On single-phase tasks (Figure 12), Chiron reduces JCT by an average of 5.22% over the best baseline on four of six tasks, and is near parity on the other two, skewed workloads that JuiceFS-Adaptive already suits. Chiron thus yields generally effective caching across AI workloads.
Figure 11. Five-policy comparison on multi-phase workflows W1–W3 in Table 2. No-cache has zero CHR in all three workflows and is omitted from the CHR plot. JCT is normalized to JuiceFS within each workflow.
Figure 12. JCT on single-phase tasks T1–T6 in Table 2, normalized to JuiceFS for each task.
Microscopic Performance Deep Dive
We next examine prefetching and eviction separately on the Stable Diffusion 1.5 workflow (W3 in Table 2), with the setup of § 8.1 unless otherwise specified.
Eviction performance. We compare Chiron with LRU, LeCaR, and JuiceFS 2-random, with prefetching disabled and the same cache size in all arms. Chiron has the highest CHR and reduces JCT by 17.65% relative to the best baseline (Figure 13a), which confirms that its eviction policy is better than static or low-end adaptive methods.
Figure 13. Eviction and prefetch comparisons on the Stable Diffusion 1.5 workflow (W3 in Table 2).
Prefetching performance. We compare Chiron with sequential prefetching at fixed depth, sequential prefetching at adaptive depth [43], and SFP ([84]), with the same cache size and LRU eviction in all arms. Chiron outperforms all three, improving CHR by
31.11% (24.19% versus 18.45%) and JCT by 11.63% (Figure 13b).
Ablation Study
§ 5 adopts multiple techniques to make Chiron fast, cheap, and effective, such as structured statistics, confidence gating, and feedback reflection. We evaluate each of them here, on diverse workflows for representativeness.
Necessity of structured statistics. We compare Chiron with a variant that has no structured statistics in its prompt, on the Stable Diffusion 1.5 workflow. Without structured statistics, the F1 score of pattern diagnosis degrades by 45% and CHR falls by 43% (Figure 14a), so these statistics improve both pattern recognition and end-to-end performance.

Figure 14. Ablation studies. Structured statistics are evaluated on the Stable Diffusion 1.5 workflow (W3 in Table 2), normalized to full Chiron; confidence gating and feedback reflection are evaluated on the ResNet-50 workflow trace.
Necessity of confidence gate. On the ResNet-50 workflow5, enabling the gate raises CHR by 12.60% in relative terms and reduces JCT from 1,838.5 to 1,799.4 s (Figure 14b). Confidence gating is thus necessary to handle unstable LLM decisions.
Necessity of feedback reflection. Also on the ResNet-50 workflow, we compare LLM calls once per 30 s without reflection and once per 45 s with reflection, which allows early calls triggered by feedback. With reflection, Chiron attains comparable CHR and JCT with 28.1% fewer tokens (Figure 14c).
Sensitivity Analysis to LLM Capability
We replay the ResNet-50 workflow with six models: Qwen3.8-Flash, Qwen3.7-Flash, Qwen3.5-Flash, Qwen3.5-9B, Qwen3.5-4B, and R1-Distill-Qwen-1.5B. Chiron yields similar caching performance from Qwen3.8-Flash to Qwen3.5-9B, with JCT between 4.61k and 4.70k s, yet degrades markedly at 4B and below, with JCT of 5.37k s for Qwen3.5-4B and 6.21k s for R1-Distill-Qwen-1.5B (Figure 15). A modest LLM, larger than 4B, is thus desirable. Token cost also grows monotonically as the LLM shrinks, from 276k to 607k tokens, because weaker models need more thinking tokens.
Figure 15. Diagnosis-model sensitivity on the ResNet-50 workflow replay.
Large-Scale Simulation
We build an event-driven cache simulator to assess Chiron at large scale, simulating the ResNet-50 workflow on an expanded 1 TB ImageNet dataset. This matches the TB-scale sizes of widely used AI datasets such as the 1.31 TB full-resolution ImageNet-21K archive (Alibaba-MIIL) and the approximately 10 TB LAION-400M WebDataset [48]. Each access costs 2 ms on a cache hit and 250 ms on a miss, and cache sizes are 10%, 25%, 50%, and 85% of the dataset. In a fidelity test, the simulator differs from end-to-end measurements by 2.82 percentage points of CHR on average and 4.42 at most.
According to Figure 16, Chiron has the highest CHR and lowest simulated JCT at every cache size, improving CHR by 3.61–8.79 percentage points and reducing JCT by 5.72–16.69% over the best baseline. The average LLM call delay stays below 1.3 s and is not on the critical path. With bounded input and output lengths and asynchronous decisions, both independent of dataset size, Chiron can scale to production scenarios.
Figure 16. Policy comparison in the large-scale dataset simulation. Cache capacity is expressed as a fraction of dataset bytes. JCT is normalized to JuiceFS at each capacity.
Related Work
Workload-aware caching for AI workloads. AI storage systems exploit training structure to reduce data-loading costs [31, 37, 80, 87]. Quiver [47] exploits sample substitutability, CoorDL [61] retains samples across shuffled epochs, and NoPFS [24] predicts accesses from sampling seeds to coordinate caching and prefetching. SHADE [44] uses sample importance to guide sampling and caching, and SiloD [93] co-designs caching and cluster scheduling. Chiron instead diagnoses observed file accesses to select policies across workflow phases, without sampling seeds or sample-importance scores.
Adaptive and learning-based cache management. Adaptive policies infer useful cache behavior from access history [38, 39, 10, 89]. ARC [58] balances recency and frequency, LeCaR [83] and CACHEUS [73] learn to combine replacement experts, LRB [77] approximates Belady-style eviction, and SFP [84] learns file-access patterns for prefetching. Learning-Augmented Heuristics [86] also select cache parameters asynchronously while deterministic heuristics process requests; Chiron shares this separation, but uses LLM-based pattern diagnosis to coordinate prefetching, admission, and eviction.
LLM-assisted systems management. LLMs increasingly support system diagnosis, tuning, and runtime decisions [76, 17, 15, 35]. D-Bot [95] uses LLMs for database diagnosis, GPTuner [49] extracts documentation knowledge to guide tuning, λ-Tune [28] generates and evaluates database configurations, NetLLM (Wu et al., 2024) adapts LLMs to networking tasks, and An Expert in Residence [55] explores online OS tuning. PF-LLM (Xu et al., 2026) separates offline assembly analysis from runtime hardware prefetching through LLM-generated hints; Chiron extends this division to online file caching, using reusable diagnoses and cache feedback to track changing access patterns.
Conclusion
AI workloads need a cache that adapts to diverse and changing access patterns without user hints. Chiron provides it through advisory switching control: an LLM asynchronously diagnoses the access pattern and recommends a policy, while a deterministic harness gates this advice, executes expert-designed policies, and feeds cache outcomes back for review. Implemented atop JuiceFS, Chiron reduces JCT by 14.3–41.2% over the best baseline on multi-phase workflows. The same division of work may suit other systems whose decisions are too frequent for an LLM to make directly.
Alibaba-MIIL. ImageNet-21K-P: Dataset preprocessing instructions. URL https://github.com/Alibaba-MIIL/ImageNet21K/blob/main/dataset_preprocessing/processing_instructi
Diagnosis Prompt and Input Fields
This appendix provides the full system prompt, an example runtime prompt, and definitions of its fields to accompany § 5.2.1 and Figure 8. The prompt and its statistics fields use the terminology of the main text and Figure 8.
Classify an ordered block-ID sequence as exactly one pattern: "sequential", "random", or "
skewed".
Definitions:
- sequential: ordered traversal or a stable stride; it need not use only ±1 steps.
- random: broadly dispersed accesses without stable order or concentrated reuse.
- skewed: repeated accesses concentrated on a comparatively small subset of IDs.
Use access sequence and structured statistics jointly to assess the candidate
patterns; no single statistic is a mandatory classification rule. Compare
full-window and tail-window statistics to distinguish recent behavior from
older accesses during a phase transition. Use review context to identify the
reason for review, the previous diagnosis, and potentially outdated accesses.
Treat this context as a reason to reconsider the diagnosis, not as evidence
for a pattern, and do not assume the previous diagnosis remains correct.
Normalize confidence to a score between 0 and 1 inclusive, not a percentage.
The confidence score expresses a model judgment, not a calibrated probability.
r is optional debugging output.
Return only valid JSON:
{"p":"sequential|random|skewed","c":<number from 0 to 1>,"r":"optional debug reason"}
Runtime prompt
The runtime prompt below uses the illustrative sequence and a routine review of a previously accepted random pattern. Only the sequence is abbreviated; all statistics and context fields are shown.
Access sequence: [0,2,4,...,794,796,798]
Structured statistics:
{"window": {
"length": 400, "unique_ratio": 1.0, "access_entropy": 1.0,
"gini": 0.0, "top_5_access": 0.013, "reuse_ratio": 0.0,
"short_reuse_ratio": 0.0, "mean_reuse_distance": 400.0,
"adjacent_steps": {
"zero_ratio": 0.0, "pm1_ratio": 0.0,
"small_jump_ratio": 1.0, "large_jump_ratio": 0.0,
"dominant_steps": [{"delta": 2, "count": 399}]}},
"recent_tail": {
"length": 200, "unique_ratio": 1.0, "access_entropy": 1.0,
"gini": 0.0, "top_5_access": 0.025, "reuse_ratio": 0.0,
"short_reuse_ratio": 0.0, "mean_reuse_distance": 200.0,
"adjacent_steps": {
"zero_ratio": 0.0, "pm1_ratio": 0.0,
"small_jump_ratio": 1.0, "large_jump_ratio": 0.0,
"dominant_steps": [{"delta": 2, "count": 199}]}}}
Review context:
{"review_trigger": "periodic_retry", "previous_control_state": "stable",
"previous_committed_pattern": "random", "older_prefix_may_be_stale": false,
"recent_tail_length": 200}
Runtime prompt field definitions
The runtime prompt consists of an ordered access sequence followed by two JSON objects containing structured statistics and review context. The three groups are labeled Access sequence, Structured statistics, and Review context, as in Figure 8. Table 3 defines the structured statistics; the notation used in its formulas is specified below.
| Field | Meaning | Definition |
Scalar fields in both window and recent_tail | ||
length | Number of accesses | |
unique_ratio | Distinct IDs per access | |
access_entropy | Diversity of ID frequencies | if ; 0 otherwise |
gini | Inequality of ID frequencies | |
top_5_access | Share of accesses to the five most frequent IDs | |
reuse_ratio | Share of accesses that repeat an ID | |
short_reuse_ratio | Share of repeats with short gaps | |
mean_reuse_distance | Mean gap between repeated IDs | if ; otherwise |
Fields inside adjacent_steps in either summary, for | ||
zero_ratio | Consecutive repeated IDs | |
pm1_ratio | Unit-step traversal | |
small_jump_ratio | Small nonzero ID steps | |
large_jump_ratio | Large ID steps | |
dominant_steps | Most frequent ID steps | Up to five pairs with largest , ordered by decreasing count; ties follow first appearance. |
delta,count | Step value and its frequency | The two members of each pair: and . |
Table 3 (PDF p. 17). Table 3. Structured statistics computed from the access sequence. Ratios with zero denominators are 0 unless an explicit fallback is shown. Ratios, entropy, and Gini are rounded to three decimal places; mean reuse gap is rounded to one. Lengths, step differences, and counts are integers.
Within the statistics object, window and recent_tail correspond to full_window_statistics and tail_window_statistics.
Let denote Access sequence, and let
The window object summarizes ; recent_tail summarizes with the same scalar and adjacent-step fields, computed within each sequence. For either analyzed sequence , define
Write for ascending counts and for descending counts. For repeated accesses, let
All reuse gaps count sequence positions within the analyzed window, rather than elapsed time or byte distance. For adjacent steps, let and .
For review context, let denote the diagnosis trigger, the controller state when the snapshot is taken, and the previously committed pattern. The state defaults to default when unavailable; the pattern is null when no valid previous pattern is available. Let . Table 4 defines the context fields; is the tail length defined above.
| Review-context field | Meaning | Value |
review_trigger | Reason for requesting diagnosis | chr_anomaly if ; periodic_retry otherwise |
previous_control_state | Earlier controller state | |
previous_committed_pattern | Earlier committed pattern | |
older_prefix_may_be_stale | Possible outdated access prefix | |
recent_tail_length | Number of tail accesses |
Table 4. Review-context fields in Review context. The stale-prefix flag is a Boolean warning, not an observed phase boundary.
Sensitivity to the CHR reflection threshold
A lower CHR deviation threshold makes feedback reflection more sensitive: it can request more diagnoses and react sooner to pattern changes, at the cost of additional diagnosis tokens. To examine this tradeoff, we replay the ResNet-50 workflow W1 from § 8.1. We set both the CHR drop and increase thresholds relative to the previous EWMA to 0.40, 0.20, and 0.10, while holding the other settings fixed.
Figure 17 shows that lowering the threshold from 0.40 to 0.10 increases feedback-triggered diagnoses from 0 to 20, while mean recovery time across pattern transitions falls from 38.7 to 23.4 s. Total diagnosis tokens increase from 83.7k to 96.1k. The data indicates that a more sensitive feedback gate can improve adaptation at the expense of diagnosis cost. However, CHR remains nearly identical across the settings, ranging from 69% to 70%. This is because, as the dataset and its phases grow longer, the accesses made before a periodic diagnosis corrects the pattern account for a smaller fraction of the workflow. The resulting difference in cache hits is therefore diluted by the many accesses served after the policy has adapted.
Figure 17. Sensitivity to the CHR reflection threshold on the ResNet-50 workflow. Both the drop and increase thresholds take the value shown. Recovery time is the mean time from a phase boundary until an accepted diagnosis commits the matching pattern.
References
- [3]Alluxio. Caching: Alluxio storage overview, 2026a. URL https://documentation.alluxio.io/os-en/core-services/caching. Retrieved on September 23, 2026.
- [4]Alluxio. RedNote accelerates model training and distribution with Alluxio, 2026b. URL https://www.alluxio.io/customer-stories/rednote. Retrieved on September 23, 2026.
- [6]Amazon Web Services. Amazon S3, 2026. URL https://aws.amazon.com/s3/. Retrieved on September 23, 2026.
- [7]Ananthanarayanan, G., Ghodsi, A., Warfield, A., Borthakur, D., Kandula, S., Shenker, S., and Stoica, I. PACMan: Coordinated memory caching for parallel jobs. In 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12), pp. 267–280, 2012.
- [8]Azure. Access data in a job, 2026. URL https://learn.microsoft.com/en-us/azure/machine-learning/how-to-read-write-data-v2. Retrieved on September 23, 2026.
- [9]Baevski, A., Zhou, Y., Mohamed, A., and Auli, M. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, volume 33, 2020.arxiv.org/abs/2006.11477
- [10]Beckmann, N., Chen, H., and Cidon, A. LHD: Improving cache hit rate by maximizing hit density. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18), pp. 389–403, 2018.
- [11]Bodner, T., Radig, T., Justen, D., Ritter, D., and Rabl, T. An empirical evaluation of serverless cloud infrastructure for large-scale data processing. In Proceedings of the 28th International Conference on Extending Database Technology (EDBT), pp. 935–948, 2025. doi: 10.48786/e dbt.2025.76.arxiv.org/abs/2501.07771
- [12]Bojarski, M., Testa, D. D., Daworowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller U., Zhang, J., Zhang, X., Zhao, J., and Zieba, K. End to end learning for self-driving cars, 2016. URL https://arxiv.org/abs/1604.07316.
- [13]Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pp. 1877–1901, 2020.arxiv.org/abs/2005.14165
- [15]Chen, H., Wen, Y., Cheng, L., Kuang, S., Liu, Y., Li, W., Li, L., Zhang, R., Song, X., Li, W., Guo, Q., and Chen, Y. AutoOS: Make your OS more powerful by exploiting large language models. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=Rp8R9C0St h.
- [16]Chen, T.-F. and Baer, J.-L. Effective hardware-based data prefetching for high-performance processors. IEEE transactions on computers, 44(5):609–623, 1995.DOI
- [17]Cummins, C., Seeker, V., Grubisic, D., Roziere, B., Gehring, J., Synnaeve, G., and Leather, H. Meta large language model compiler: Foundation models of compiler optimization. arXiv preprint arXiv:2407.02524, 2024.arxiv.org/abs/2407.02524
- [19]DeepSeek-AI et al. DeepSeek-V4: Towards highly efficient million-token context intelligence, 2026. URL https://arxiv.org/abs/2606.19348.
- [20]Denning, P. J. The working set model for program behavior. Communications of the ACM, 11(5):323–333, 1968.
- [21]Desai, D., Ghita, L., and Boier, I. Synthetic data generation for financial AI research with NVIDIA NeMo. NVIDIA Technical Blog, July 2026. URL https://developer.nvidia.com/blog/synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo/. Retrieved on October 12, 2026.
- [22]Di Natale, F., Bermudez, E., Casagrande, A., Kotila, M., Komali, H., and Kudleppanavar, G. Benchmarking LLM inference at scale with AIPerf. NVIDIA Technical Blog, September 2026. URL https://developer.nvidia.com/blog/benchmarking-llm-infer ence=at-scale-with-aiperf/. Retrieved on October 13, 2026.
- [23]Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021. URL https://arxiv.org/abs/2010.11929.
- [24]Dryden, N., Böhringer, R., Ben-Nun, T., and Hoefler, T. Clairvoyant prefetching for distributed machine learning I/O. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC), pp. 1–15, 2021. doi: 10.1145/3458817.3476181.DOI
- [25]Floratou, A., Megiddo, N., Potti, N., Özcan, F., Kale, U., and Schmitz-Hermes, J. Adaptive caching in Big SQL using the HDFS cache. In Proceedings of the Seventh ACM Symposium on Cloud Computing, pp. 321–333, 2016.DOI
- [27]Garg, S., Zhang, J., Pitchumani, R., Parashar, M., Xie, B., and Kannan, S. CrossPrefetch: Accelerating I/O prefetching for modern storage. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, pp. 102–116, 2024.DOI
- [28]Giannakouris, V. and Trummer, I. λ-Tune: Harnessing large language models for automated database system tuning. Proceedings of the ACM on Management of Data, 3(1): 1–26, 2025.DOI
- [30]Gowtham, Rupesh, S., Kumar, S., Saravanan, and Chaithanya, V. FLUX: Data worth training on, 2026. URL https://arxiv.org/abs/2603.13972.
- [31]Graur, D., Aymon, D., Kluser, D., Albrici, T., Thekkath, C. A., and Klimovic, A. Cachew: Machine learning input data processing as a service. In 2022 USENIX Annual Technical Conference (USENIX ATC 22), pp. 689–706. USENIX Association, 2022.
- [35]Huang, X., Li, H., Zhang, J., Zhao, X., Yao, Z., Li, Y., Zhang, T., Chen, J., Chen, H., and Li, C. E2ETune: End-to-end knob tuning via fine-tuned generative language model. arXiv preprint arXiv:2404.11581, 2024.arxiv.org/abs/2404.11581
- [37]Jha, K. V., Pandey, S., Annavaram, M., and Basu, A. Hy-Cache: Hybrid caching for accelerating DNN input pre-processing pipelines. In 2025 USENIX Annual Technical Conference (USENIX ATC 25), pp. 433–448. USENIX Association, 2025.
- [38]Jiang, S. and Zhang, X. LIRS: An efficient low interference recency set replacement policy to improve buffer cache performance. In Proceedings of the 2002 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, pp. 31–42, 2002. doi: 10.1145/511334.511340.DOI
- [39]Jiang, S., Chen, F., and Zhang, X. CLOCK-Pro: An effective improvement of the CLOCK replacement. In 2005 USENIX Annual Technical Conference (USENIX ATC 05), pp. 323–336. USENIX Association, 2005.
- [40]Jin, C., Zhang, Z., Jiang, X., Liu, F., Liu, S., Liu, X., and Jin, X. RAGCache: Efficient knowledge caching for retrieval-augmented generation. ACM Transactions on Computer Systems, 44(1):2:1–2:27, 2025. doi: 10.1145/3768628.DOI
- [42]JuiceFS. Achieving elastic throughput in the cloud with a distributed file system to boost AI training, 2023. URL https://juicefs.com/en/blog/solutions/accelerate-ai-training-flexible-elastic-throughput-cloud. Retrieved on September 23, 2026.
- [43]JuiceFS. Cache: JuiceFS community edition documentation, 2026. URL https://juicefs.com/docs/community/guide/cache/. Retrieved on September 23, 2026.
- [44]Khan, R. I. S., Yazdani, A. H., Fu, Y., Paul, A. K., Ji, B., Jian, X., Cheng, Y., and Butt, A. R. SHADE: Enable fundamental cacheability for distributed deep learning training. In 21st USENIX Conference on File and Storage Technologies (FAST 23), pp. 135–152, 2023.
- [45]Kimi Team. Kimi K3: Open frontier intelligence, 2026. URL https://arxiv.org/abs/2607.24653.
- [46]Kuchnik, M., Klimovic, A., Simsa, J., Smith, V., and Amvrosiadis, G. Plumber: Diagnosing and removing performance bottlenecks in machine learning data pipelines. In Proceedings of Machine Learning and Systems (MLSys), 2022.arxiv.org/abs/2111.04131
- [47]Kumar, A. V. and Sivathanu, M. Quiver: An informed storage cache for deep learning. In 18th USENIX Conference on File and Storage Technologies (FAST 20), pp. 283–296. USENIX Association, 2020.
- [48]LAION. LAION-400M Open Dataset, 2021. URL https://laion.ai/blog/laion-400-open-dataset/. Image-caption WebDataset size: 10 TB. Accessed October 27, 2026.
- [49]Lao, J., Wang, Y., Li, Y., Wang, J., Zhang, Y., Cheng, Z., Chen, W., Tang, M., and Wang, J. GPTuner: A manual-reading database tuning system via GPT-guided Bayesian optimization. Proceedings of the VLDB Endowment, 17 (8):1939–1952, 2024. doi: 10.14778/3659437.3659449.DOI
- [50]Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., tau Yih, W., Rocktäschel, T., Riedel, S., and Kiela, D. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems, volume 33, pp. 9459–9474, 2020.arxiv.org/abs/2005.11401
- [55]Liargkovas, G., Jabrailov, V., Franke, H., and Kaffes, K. An expert in residence: LLM agents for always-on operating system tuning. Machine Learning for Systems, 2025.
- [56]Liberzon, D. Switching in Systems and Control: Foundations & Applications. Birkhäuser, Boston, MA, 2003. doi: 10.1007/978-1-4612-0017-8.DOI
- [57]Mattson, R. L., Gecsei, J., Slutz, D. R., and Traiger, I. L. Evaluation techniques for storage hierarchies. IBM Systems Journal, 9(2):78–117, 1970. doi: 10.1147/sj.92.0078.DOI
- [58]Megiddo, N. and Modha, D. S. ARC: A self-tuning, low overhead replacement cache. In 2nd USENIX Conference on File and Storage Technologies (FAST 03). USENIX Association, 2003.
- [60]Microsoft. Azure Blob Storage, 2026. URL https://azure.microsoft.com/en-us/products/storage/blobs/. Retrieved on October 27, 2026.
- [61]Mohan, J., Phanishayee, A., Raniwala, A., and Chidambaram, V. Analyzing and mitigating data stalls in DNN training. Proceedings of the VLDB Endowment, 14 (5):771–784, 2021. doi: 10.14778/3446095.3446100.DOI
- [62]Morse, A. S. Supervisory control of families of linear set-point controllers—part I: Exact matching. IEEE Transactions on Automatic Control, 41(10):1413–1431, 1996. doi: 10.1109/9.539424.DOI
- [63]Murray, D. G., Simsa, J., Klimovic, A., and Indyk, I. tf.data: A machine learning data processing framework. Proceedings of the VLDB Endowment, 14(12):2945–2958, 2021.
- [65]Naumov, M., Mudigere, D., Shi, H.-J. M., Huang, J., Sundaraman, N., Park, J., Wang, X., Gupta, U., Wu, C.-J., Azzolini, A. G., Dzhulgakov, D., Mallevich, A., Cherniavskii, I., Lu, Y., Krishnamoorthi, R., Yu, A., Kondratenko, V., Pereira, S., Chen, X., Chen, W., Rao, V., Jia, B., Xiong, L., and Smelyanskiy, M. Deep learning recommendation model for personalization and recommendation systems, 2019.arxiv.org/abs/1906.00091
- [67]Penedo, G., Kydlíček, H., Sabolčec, V., Messmer, B., Foroutan, N., Kargaran, A. H., Raffel, C., Jaggi, M., Werra, L. V., and Wolf, T. FineWeb2: One pipeline to scale them all—adapting pre-training data processing to every language, 2025. URL https://arxiv.org/abs/2506.20920.
- [68]Qureshi, M. K., Jaleel, A., Patt, Y. N., Steely, Jr., S. C., and Emer, J. Adaptive insertion policies for high performance caching. ACM SIGARCH Computer Architecture News, 35(2):381–391, 2007. doi: 10.1145/1273440.1250709.DOI
- [71]Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21 (140):1–67, 2020. URL https://www.jmlr.org/papers/volume21/20-074.html.arxiv.org/abs/1910.10683
- [73]Rodriguez, L. V., Yusuf, F., Lyons, S., Paz, E., Rangaswami, R., Liu, J., Zhao, M., and Narasimhan, G. Learning cache replacement with CACHEUS. In 19th USENIX Conference on File and Storage Technologies (FAST 21), pp. 341–354. USENIX Association, 2021.
- [74]Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., and Aberman, K. DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22500–22510, 2023.arxiv.org/abs/2208.12242
- [75]Seo, D., Oh, H.-S., and Jung, Y. Wav2KWS: Transfer learning from speech representations for keyword spotting. IEEE Access, 9:80682–80691, 2021. doi: 10.1109/ACCESS.2021.3078715.DOI
- [76]Singh, V., Vaidya, K. E., Kumar, V. B., Khosla, S., Narayanaswamy, M., Gangadharaiah, R., and Kraska, T. Panda: Performance debugging for databases using LLM agents. URL https://www.amazon.science/publications/panda-performance-debugging-for-databases-using-llm-agents, 2024.
- [77]Song, Z., Berger, D. S., Li, K., and Lloyd, W. Learning relaxed Belady for content distribution network caching. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20), pp. 529–544. USENIX Association, 2020. URL https://www.usenix.org/conference/nsdi20/presentation/song.
- [78]Srivastava, A. NVIDIA accelerates inference on Meta Llama 4 Scout and Maverick. NVIDIA Technical Blog, April 2025. URL https://developer.nvidia.com/blog/nvidia-accelerates-inference-on-meta-llama-4-scout-and-maverick/. Retrieved on October 12, 2026.
- [80]Sun, B., Yu, X., Zhang, C., Tian, J., Jin, S., Iskra, K., Zhou, T., Bicer, T., Beckman, P., and Tao, D. SOLAR: A highly optimized data loading framework for distributed training of CNN-based scientific surrogates. arXiv preprint arXiv:2211.00224, 2022.arxiv.org/abs/2211.00224
- [81]Takamoto, M., Praditia, T., Leiteritz, R., MacKinlay, D., Alesiani, F., Pflüger, D., and Niepert, M. PDEBench: An extensive benchmark for scientific machine learning. In Advances in Neural Information Processing Systems, volume 35, 2022.arxiv.org/abs/2210.07182
- [82]Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 30, 2017.arxiv.org/abs/1706.03762
- [83]Vietri, G., Rodriguez, L. V., Martinez, W. A., Lyons, S., Liu, J., Rangaswami, R., Zhao, M., and Narasimhan, G. Driving cache replacement with ML-based LeCaR. In 10th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 18). USENIX Association, 2018.
- [84]Wang, H., Luo, L., Shi, L., Li, C., Xue, C. J., Zhuge, Q., and Sha, E. H.-M. SFP: Smart file-aware prefetching for flash based storage systems. In Proceedings of the 2021 Great Lakes Symposium on VLSI (GLSVLSI), 2021. doi: 10.1145/3453688.3461492.
- [86]Xia, H., Nixon, W., Marthen, B. D., Bhandari, P., and Yang, J. Learning-Augmented heuristics: Simple yet smart, robust and interpretable cache eviction. In 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI 26), pp. 2241–2259, Seattle, WA, July 2026. USENIX Association. ISBN 978-1-939133-55-7. URL https://www.usenix.org/conference/osdi26/presentation/xia.arxiv.org/abs/2608.27975
- [87]Xie, J., Xu, J., Wang, G., Yao, Y., Li, Z., Cao, C., and Tong, H. A deep learning dataloader with shared data preparation. Advances in Neural Information Processing Systems, 35:17146–17156, 2022.DOI
- [89]Yang, J., Mao, Z., Yue, Y., and Rashmi, K. GL-Cache: Group-level learning for efficient and high-performance caching. In 21st USENIX Conference on File and Storage Technologies (FAST 23), pp. 115–134, 2023a.
- [90]Yang, J., Zhang, Y., Qiu, Z., Yue, Y., and Rashmi, K. V. FIFO queues are all you need for cache eviction. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP), pp. 130–149, 2023b. doi: 10.1145/3600006.3613147.DOI
- [91]Zhang, K., Li, B., Zhang, P., Pu, F., Cahyono, J. A., Hu, K., Liu, S., Zhang, Y., Yang, J., Li, C., and Liu, Z. LMMs-Eval: Reality check on the evaluation of large multimodal models, 2024a. URL https://arxiv.org/abs/2407.12772.
- [92]Zhang, Y., Yang, J., Yue, Y., Vigfusson, Y., and Rashmi, K. V. SIEVE is simpler than LRU: An efficient turnkey eviction algorithm for web caches. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), pp. 1229–1246. USENIX Association, 2024b. URL https://www.usenix.org/conference/nsdi24/presentation/zhang-yazhuo.
- [93]Zhao, H., Han, Z., Yang, Z., Zhang, Q., Li, M., Yang, F., Zhang, Q., Li, B., Yang, Y., Qiu, L., Zhang, L., and Zhou, L. SiloD: A co-design of caching and scheduling for deep learning clusters. In Proceedings of the Eighteenth European Conference on Computer Systems, EuroSys, 2023.DOI
- [94]Zheng, B., Mao, B., and Pan, Z. Thousand-node Alluxio cluster powers game AI platform: A production case study from Tencent, 2022. URL https://www.alluxio.io/blog/thousand-node-alluxio-cluster-powers-game-ai-platform-production-case-study-from-tencent. Retrieved on September 23, 2026.
- [95]Zhou, X., Li, G., Sun, Z., Liu, Z., Chen, W., Wu, J., Liu, J., Feng, R., and Zeng, G. D-Bot: Database diagnosis system using large language models. Proceedings of the VLDB Endowment, 17(10):2514–2527, 2024. doi: 10.14778/3675034.3675043.DOI
- [96]Zhou, X., Hao, X., Yu, X., and Stonebraker, M. Tiered-Indexing: Optimizing access methods for skew. The VLDB Journal, 34(4):45, 2025.DOI














