WorksComputer Science
YUL: Bringing Software Supply Chain Security to Coding Agents
Abstract
Coding agents choose and declare third-party dependencies on their own. They often declare a stale version that they recall from their training data. A stale dependency exposes the project to known vulnerabilities that the latest release already fixes. We present YUL, a design for coding agent harnesses that ensures that no stale dependencies are ever used. It does so by checking every dependency declaration against the package registry. If a dependency is stale or does not exist, YUL blocks the usage and tells the coding agent which version to use, so the coding agent retries with the fix. We evaluate YUL on 60 tasks across six ecosystems with Claude Code running Claude Opus 5.5 and OpenCode running DeepSeek V4.1 Flash. Without YUL, Opus declares a stale version in 29% of runs and DeepSeek in 39%, and both declare a stale version in every GitHub Actions run. With YUL, the coding agents end on the latest release in 93% of stale Opus runs and in 96% of stale DeepSeek runs. YUL thus brings the best practices of software supply chain security to coding agents.
Introduction
Coding agents solve programming tasks on their own. A developer describes a task in natural language, and the coding agent plans a solution, edits files, runs commands, and tests the result until the task is complete. Along the way, coding agents do an essential software engineering task: they choose third-party dependencies to reuse.
When a task needs a dependency, the coding agent declares it in a manifest, such as package.json or pom.xml, and a version is taken. This dependency version can be vulnerable or even malicious, and it then puts the whole project at risk [1]. In this paper, we address the problem that coding agents have no principled way of using dependencies.
First and foremost, coding agents may use stale dependencies. A dependency is stale when its declaration does not resolve to the latest release of its package. This happens for instance when the coding agent recalls a version from the training data, which ends at a knowledge cutoff. Stale dependencies carry a security cost, since systems with outdated dependencies are four times as likely to have security issues as systems that are up to date [2]. The OpenSSF guide for AI code assistants recommends a prompt that tells the coding agent to “prefer the latest stable release” [3]. This is weak: prompting alone cannot enforce that only latest versions are used and agents may still use a stale version.
To solve this problem, we present YUL, a coding agent design pattern that guarantees that only latest dependency versions are used. YUL looks up the latest release in the package registry for each dependency that a coding agent wants to use. If the coding agent declares a stale version, YUL blocks the write and reports the latest release, so the coding agent adapts its behavior.
To evaluate YUL, we build a benchmark of 60 dependency-aware coding tasks across six ecosystems, namely Maven, GitHub Actions, PyPI, npm, Go modules, and Cargo. A task asks for a feature but never mentions a dependency, so that the coding agent can decide on its own what to reuse and which version to pick. We run each task three times through two coding agents, Claude Code with Claude Opus 5.5 and OpenCode with DeepSeek V4.1 Flash, and measure how often they declare a stale version on their own (RQ1). Finally, we measure how effectively YUL mitigates the stale versions (RQ2).
Our results show that YUL is effective in mitigating the stale dependency problem. Without YUL, Opus declares a stale version in 42 of 146 valid runs (29%) and DeepSeek in 54 of 139 (39%) cases. With YUL, the coding agents end on the latest release in 39 of 42 stale Opus runs (93%) and in 52 of 54 stale DeepSeek runs (96%). The remaining cases where YUL does not prevent the usage of stale dependencies ties to corner cases of environment management and dependency resolution.
To summarize, this paper makes the following contributions.
An empirical study of how often coding agents declare and use stale dependency versions across six ecosystems.
YUL, a publicly available system for coding agents that blocks stale dependency declarations, available at https://github.com/chains-project/yul.
An evaluation of YUL on 60 tasks and 720 runs, which shows that YUL mitigates 93% and 96% of the stale dependency cases.
A benchmark of 60 dependency tasks and the transcripts of all runs, available at https://github.com/chains-project/yul-benchmark.
Sections II to IX present, in order, the background, the design of YUL, the methodology, the results, a discussion, threats to validity, related work, and the conclusion.
Background
Coding Agents
A coding agent combines an LLM with a harness, which runs the LLM in a loop with specific prompting and format expectations. Examples of coding agents include Claude Code or OpenCode. When a coding agent is given a coding task, it leverages the LLM to plan, write, and test code until the task is complete.
A harness lets users run their own commands, called hooks, at fixed points of this loop [4]. Hooks are not prompting, they give deterministic control, by being run in the loop instead of relying on the judgment of the LLM to run them [5]. For example, developers can use hooks to format code after each edit by the LLM. A hook that runs before a tool call can allow the call or block it. When the hook blocks a call, the LLM receives the message of the hook instead of the result of the call, and the coding agent can retry with a corrected call. There are different ways of implementing hooks, Claude Code supports external programs as hooks, while OpenCode offers a plugin system [6].
Software Supply Chain Security
Virtually all software projects reuse third-party packages, which form the software supply chain of the project [7]. A project declares these packages as dependencies in a manifest, such as package.json for npm or pom.xml for Maven, and the package manager downloads them from the registry of the ecosystem.
Each dependency is code that the developers of the project do not write. If the dependency is vulnerable or malicious, it can put the whole project at risk, subject to a software supply chain attack. Software supply chain security is about practices and tooling to protect against such dependency risk [8].
Dependency updates are a particular risk surface. An update brings a project the latest features and fixes of its packages, including fixes for known vulnerabilities. A missed update keeps these vulnerabilities in the project, even though a fix already exists. We call a dependency stale when its declaration does not fetch its latest release. The latest release of a package is the most recent stable release that its registry reports. For example, requests==2.31.0 is stale if the latest release is 2.34.2, while requests>=2.31.0 is not stale, because it permits the installation of any latest release. A stale dependency is a security risk, because it exposes the project to vulnerabilities that are known and exploited [2]. For this reason, using automated update bots such as Dependabot and Renovate is considered a best practice [9].
Yet update bots leave a huge gap. A bot runs on the repository on a schedule, so it only sees a stale dependency after a developer pushes it. Between the moment a developer installs a stale dependency on their machine and the moment the update of the bot is merged, even automatically, the project builds and runs on the stale version. Cooldowns further widen this, because they delay each update of the bot by days [10]. Coding agent make the problem even more acute, because they write the stale version into the manifest and build and run code on top of it right away. This is exactly the gap that we address in this paper: the window in which stale dependencies are downloaded and executed.
yul Design
Overview
We propose a design in order to guarantee that coding agents do not use stale dependencies. Our prototype, YUL, implements this design as an extension to an existing coding agent harness. The core idea of YUL is to intercept stale dependencies as early as possible, as soon as they come out of the language model.
To guarantee the absence of stale dependencies, YUL works in six steps, as Figure 1 shows. (1) A developer prompts a coding agent to perform a programming task. The coding agent identifies the need for a library, and thus updates a manifest to import the library. (2) YUL intercepts the manifest write before it reaches disk and analyzes it. (3) YUL looks up the latest release of each dependency in the corresponding package registry. (4) If a dependency is stale or does not exist, YUL blocks the write and tells the coding agent how to fix it. (5) This forces the coding agent to retry the write with the fix. (6) Once every dependency is up to date, YUL approves the write and lets it reach disk. As a result, YUL guarantees that no stale dependency is ever written, downloaded and executed.1
Figure 1. Overview of YUL, a tool that guarantees that coding agents never use stale dependencies.
Input
YUL receives the manifest that the coding agent is about to write. For example, in Figure 1, the coding agent wants to write a Python requirements.txt that pins numpy 1.26.4, restricts pandas to releases below 3, and declares numpyfast. Every dependency in this manifest is a problem that YUL mitigates. The pin on numpy 1.26.4 is stale, because the latest release is 2.5.4. The range on pandas is stale, because it excludes the latest release 3.0.6. The package numpyfast is an hallucination that does not exist on PyPI. A hallucinated name is a supply chain risk, because an attacker can register it and publish malicious code under it [11].
A coding agent declares a dependency in the manifest in one of two ways. A file write saves the new content of the manifest directly, as in Figure 1. A package manager command records the dependency in the manifest, from a command emitted by the coding agent, without direct file write. For example, npm install [email protected] installs React 19.3.1 and records it in package.json, even if a newer release exists. YUL intercepts both cases. For a command, it checks the version that the command pins before the shell runs it. A command without a version, such as npm install react, installs the latest release, so YUL lets it pass.
Workflow
YUL extracts every dependency from the manifest and compares it with the latest release in the package registries, such as PyPI or Maven Central. YUL blocks a dependency in three cases. (1) YUL blocks an exact pin on a release older than the latest release, such as numpy==1.26.4 when the latest release is 2.5.4, and suggests the latest release as the fix.
(2) YUL blocks stale dependencies (Section II-B) and ranges that excludes the latest release, such as pandas<3 when the latest release is 3.0.6, and suggests a range that starts at the latest release, such as >=3.0.6,<4.0.0. (3) YUL blocks a dependency that does not exist in its registry, such as numpyfast, it is an hallucination. YUL has no release to suggest for hallucination, so it simply asks the coding agent to replace it with a real package. YUL accepts every other dependency addition, which is either an exact pin on the latest release or a range that admits the latest release. For example, YUL accepts numpy==2.5.4 and pandas>=2.
Output
If YUL finds a stale or hallucinated dependency, it blocks the write and tells the coding agent how to fix each dependency. Listing 1 shows this message for the write in Figure 1. For a hallucinated dependency, YUL asks the coding agent to use a real package instead, as Listing 1a shows. For a stale pin, YUL suggests the latest release, such as numpy 2.5.4 instead of 1.26.4, as Listing 1b shows. For a stale range, YUL suggests a range that admits the latest release, such as >=3.0.6,<4.0.0 instead of <3 for pandas. Then, the coding agent reads the message and retries another write as part of the agentic loop. YUL checks every retry again, so it also blocks a retry that ignores the fix.
In Figure 1, YUL finds that numpy 1.26.4 is a stale pin, that pandas<3 is a stale range, and that numpyfast does not exist on PyPI. In that case, YUL blocks all of them as long as the LLM has not fixed all three issues. When the LLM finally uses the latest version, YUL accepts the manifest to be written to disk.
Implementation
We implement YUL as a single Go binary and integrate it into two mainstream harnesses, Claude Code and OpenCode. In Claude Code, YUL exits with status 2 to block a tool call, so Claude Code returns the message of YUL to the LLM instead of running the call [4]. In OpenCode, the opencode→yul plugin invokes the same binary and throws an error with the message of YUL, so OpenCode aborts the call [6]. YUL supports six ecosystems through seven manifests, which it parses with the git-pkgs/manifests library.
Listing 1. Message from YUL to the coding agent for the write in Figure 1.
(a) YUL response for hallucinated dependency
replace these dependencies with real packages; they don’t exist on
their registry, so they’re likely hallucinated:
numpyfast 0.3
(b) YUL response for outdated dependencies
outdated dependencies, use these versions instead:
numpy 1.26.4 → 2.5.4
these pinned ranges need attention:
pandas <3 does not allow latest 3.0.6 → update range to
>=3.0.6,<4.0.0
Experimental Methodology
Research Questions
Our evaluation answers two research questions.
RQ1. How often do coding agents propose and use a stale dependency version?
RQ2. How effectively does YUL mitigate the stale dependency problem in coding agents?
Tasks
We evaluate YUL on 60 dependency-editing tasks spanning six package ecosystems: Maven (pom.xml), GitHub Actions workflows (uses: tags), PyPI (requirements.txt or pyproject.toml), npm (package.json), Go modules (go.mod), and Cargo (Cargo.toml). Each ecosystem contributes ten tasks, drawn from that ecosystem’s most-downloaded real-world packages, for a total of 60 tasks.
Task construction. For each ecosystem, we query ecosyste.ms for the ten most-downloaded packages [12]. We take each package’s name from ecosyste.ms and use a coding agent to generate a one-line natural-language task prompt around it. Each task gives the coding agent a prompt that asks for a feature to be implemented. The task never mentions a manifest file or a dependency, in order to let the LLM guide on its own.
Task example For example, the PyPI task “Set up a new Python project for a script that does heavy numerical array and matrix computations.” leaves it to the coding agent on how to implement the feature. It may decide to reuse a dependency such as NumPy, or it may choose to implement the feature from scratch.
Task validation. We manually review the generated prompt for each task to ensure that it is meaningful and accurately reflects the corresponding package’s purpose. All tasks can be reviewed in our replication package. We also record the latest release of each package as of October 2026, which we use to evaluate the coding agents’ staleness performance.
Coding Agents
We perform every task through two coding agents, Claude Code and OpenCode. We choose Claude Code because it has native support for hooks [4] and supports frontier models which consistently stay at top of LLM leaderboards [13]. OpenCode, unlike Claude Code, supports models such as DeepSeek, whose reasoning tokens are valuable to understand coding agent’s judgment on dependency versions. We run each of the 60 tasks under two conditions, nohook (the YUL is not installed) and YUL (YUL hook is installed), three times each. This gives 180 runs per condition, 360 runs in total. We repeat each run because coding agents are non-deterministic [14], so the same prompt can lead to different dependency versions. Three repetitions show whether an outcome is stable for a task or occurs in a single run only [12, 15]. We record the full transcript and the resulting manifest for every run. We have made it publicly available at https://github.com/chains-project/yul-benchmark.
Outcome classification
We first select in which the coding agent declares the target dependency (expected) instead of implementing the required feature itself (irrelevant wrt. our goal). We classify every valid dependency addition by two outcomes, stale and mitigated.
a) Stale.: A run is considered stale if the coding agent’s own write does not use the dependency’s latest release. Stale counts from the nohook condition measure how often coding agents declare a stale version and answer RQ1.
b) Mitigated.: Mitigated counts the runs where YUL rejects a stale write and the coding agent’s retry lands on the latest release. This count measures YUL’s effectiveness at correcting stale writes and answers RQ2.
A run is mitigated if and only if 1) the coding agent writes a stale version, 2) YUL rejects it, and 3) the run ends on the latest release thanks to YUL’s feedback.
Setup
Claude Code 2.1.287 runs Claude Opus 5.5 (claude-opus-5-5), and OpenCode 1.18.34 runs DeepSeek V4.1 Flash (deepseek-flash). In the YUL condition, both use YUL 0.0.21.
Experimental Results
Descriptive Statistics
Recall, we run each of the 60 tasks under two conditions, nohook and YUL, three times each, for 180 runs per condition.
No runtime dependency at all. For Opus, the coding agent never declares the target dependency in 34 nohook and 36 YUL runs, leaving 146 and 144 valid runs. For DeepSeek, this happens in 40 nohook and 31 YUL runs, one more nohook run times out after 15 minutes, and one more YUL run stops to ask a clarifying question, leaving 139 and 148 valid runs. In all these runs, the coding agent ends without declaring any dependency. Table I reports the same breakdown for both research questions, one block of columns per question.
RQ1: How often do coding agents declare and use a stale version?
The Nohooк block of Table I answers RQ1. Tasks is the number of runs considered for that ecosystem, after exclusions. Stale is how many of those runs do not pin the target package to the latest release. Overall, Claude Opus 5.5 declares a stale version in 42 of 146 runs (29%), and DeepSeek V4.1 Flash in 54 of 139 runs (39%). For example, Claude Opus 5.5 declares a stale version in 6 of 30 Maven runs, while DeepSeek V4.1 Flash does so in 19 of 30.
Memory alone leads to a stale version.
Memory alone leads to a stale version.: GitHub Actions relies purely on memory, because it has no native package manager command. Both coding agents write a floating major tag without looking up a tag, such as
actions/checkout@v4when the latest release is v7.0.1 (as of October 2026), so all 30 runs of each coding agent are stale.
The stale Cargo runs of both coding agents also rely on memory, since they type a version instead of running cargo add. A system prompt that instructs the coding agent to always declare dependencies with cargo add can prevent these stale runs. However, a coding agent may ignore such an instruction, as DeepSeek does when it rejects YUL’s suggestion in § 5-C2.
Maven shows the largest difference between the two coding agents. Opus queries maven-metadata.xml on Maven Central before writing pom.xml in 24 of 30 runs, including for build plugins. DeepSeek queries Maven Central in only 14 of 30 runs, mostly for the Spring Boot version, and types library versions from memory without reasoning about them. For example, DeepSeek writes junit-jupiter 5.11.4 in all three runs when the latest release is 6.1.3.
Even a lookup yields a stale version when the coding agent queries a stale index or overrides the result. DeepSeek twice queries the search.maven.org search API, whose stale index lists Spring Boot 3.5.3 and Lombok 1.18.38 as the newest releases. In another run, DeepSeek sees that the latest mysql-connector-j release is 26.7.0, the first under a new numbering scheme, and still chooses very stale 9.7.0 “for predictability”.
Tooling support which ensures latest version is declared.
Tooling support which ensures latest version is declared.: Go, Cargo, npm, and PyPI have few stale runs, because the coding agents declare dependencies with
go get,cargo add,npm install/npm view, anduv add. All those tools look up the latest release in the registry and write it to the manifest. For example, both coding agents rungo
get pkg@latest instead of typing a version into go.mod. PyPI runs get the latest version thanks to uv add and in a few cases, the coding agents write an open-ended range by hand, which admits the latest release. Having coding agents using those dependency installation tools is excellent.
Some stale pins are deliberate.
Both coding agents sometimes choose an older release on purpose to satisfy a compatibility constraint. For example, both coding agents pin golang.org/x/tools v0.24, since later releases require a Go version than the one available on the machine. Newer Go version restricts the project to a newer Go toolchain hence limiting compatibility. Another example is the Kotlin project that targets JDK 7. Both coding agents pin Kotlin 1.5.32 or 1.6.21 in most runs to remain compatible with JDK 7. DeepSeek reaches this choice by building with successively older Kotlin releases, and in one run downloads a JDK 8 to build Kotlin 1.6.21.
Takeaway (RQ1). Coding agents that rely on memory usually declare a stale version. This is particularly true for GitHub Actions and Maven. Go, Cargo, npm, and PyPI have few stale runs, because the LLM has been well trained to use the appropriate package manager command or an open-ended range. However, this good practice is not enforced in any way.
RQ2: How effectively does yul mitigate stale pins?
The YUL block of Table 1 answers RQ2. Tasks and Stale repeat the RQ1 definitions for the YUL condition. Mitigated counts the runs where YUL rejects a stale write and the coding agent’s retry lands on the latest release. The Pooled column next to Mitigated divides the mitigated runs of all three repetitions by their stale runs. For example, Opus has 30 valid GitHub Actions runs with YUL, 29 of them stale, and YUL mitigates all 29, so Pooled reads 29/29 (100%).
| Ecosystem | Tasks | Nohook (RQ1) | YUL (RQ2) | |||||||||||
| Stale | Pooled | Stale | Pooled | Mitigated | Pooled | |||||||||
| R1 | R2 | R3 | R1 | R2 | R3 | R1 | R2 | R3 | ||||||
| Claude Opus 5.5 | ||||||||||||||
| Cargo | 10 | 1/9 | 1/8 | 1/9 | 3/26 (12%) | 2/9 | 1/8 | 2/9 | 5/26 | 1/2 | 1/1 | 1/2 | 3/5 (60%) | |
| GitHub Actions | 10 | 10/10 | 10/10 | 10/10 | 30/30 (100%) | 10/10 | 10/10 | 9/10 | 29/30 | 10/10 | 10/10 | 9/9 | 29/29 (100%) | |
| Go | 10 | 1/10 | 1/10 | 1/10 | 3/30 (10%) | 1/10 | 1/9 | 0/9 | 2/28 | 1/1 | 1/1 | 0/0 | 2/2 (100%) | |
| Maven | 10 | 2/10 | 2/10 | 2/10 | 6/30 (20%) | 2/10 | 2/10 | 2/10 | 6/30 | 1/2 | 2/2 | 2/2 | 5/6 (83%) | |
| npm | 10 | 0/3 | 0/3 | 0/3 | 0/9 (0%) | 0/3 | 0/3 | 0/3 | 0/9 | 0/0 | 0/0 | 0/0 | 0/0 (–) | |
| PyPI | 10 | 0/7 | 0/7 | 0/7 | 0/21 (0%) | 0/7 | 0/7 | 0/7 | 0/21 | 0/0 | 0/0 | 0/0 | 0/0 (–) | |
| All | 60 | 14/49 | 14/48 | 14/49 | 42/146 (29%) | 15/49 | 14/47 | 13/48 | 42/144 | 13/15 | 14/14 | 12/13 | 39/42 (93%) | |
| DeepSeek V4.1 Flash | ||||||||||||||
| Cargo | 10 | 1/8 | 1/8 | 0/9 | 2/25 (8%) | 1/8 | 0/9 | 1/9 | 2/26 | 1/1 | 0/0 | 1/1 | 2/2 (100%) | |
| GitHub Actions | 10 | 10/10 | 10/10 | 10/10 | 30/30 (100%) | 10/10 | 9/9 | 10/10 | 29/29 | 9/10 | 9/9 | 9/10 | 27/29 (93%) | |
| Go | 10 | 1/8 | 1/7 | 1/8 | 3/23 (13%) | 0/8 | 1/8 | 0/8 | 1/24 | 0/0 | 1/1 | 0/0 | 1/1 (100%) | |
| Maven | 10 | 6/10 | 7/10 | 6/10 | 19/30 (63%) | 9/10 | 7/10 | 6/10 | 22/30 | 9/9 | 7/7 | 6/6 | 22/22 (100%) | |
| npm | 10 | 0/3 | 0/3 | 0/2 | 0/8 (0%) | 0/5 | 0/6 | 0/4 | 0/15 | 0/0 | 0/0 | 0/0 | 0/0 (–) | |
| PyPI | 10 | 0/8 | 0/7 | 0/8 | 0/23 (0%) | 0/8 | 0/8 | 0/8 | 0/24 | 0/0 | 0/0 | 0/0 | 0/0 (–) | |
| All | 60 | 18/47 | 19/45 | 17/47 | 54/139 (39%) | 20/49 | 17/50 | 17/49 | 54/148 | 19/20 | 17/17 | 16/17 | 52/54 (96%) |
Table 1. Stale and mitigated runs per ecosystem and repetition (R1 to R3), without and with YUL. Stale counts runs where the coding agent’s own write does not pin the latest release. Mitigated counts runs where YUL rejects a stale write and the coding agent then pins the latest release. Each pooled column adds up the numerators and the denominators of the three repetitions. Dark green marks repetitions in which YUL mitigates every stale run. Light green marks repetitions in which YUL mitigates only some stale runs. Repetitions with no stale run (0/0) stay uncolored.
GitHub Actions and Maven gain the most.
GitHub Actions for both coding agents and Maven for DeepSeek have the highest stale counts in RQ1, and YUL mitigates the most runs there. In every mitigated GitHub Actions run, the coding agent replaces its floating major tag with the commit SHA that YUL suggests, such as actions/checkout@3d3c42e... # v7.0.1. Mitigated is a lower bound on the effect of YUL, because it only looks at the target dependency of each task. YUL also rejects stale versions of the other dependencies that the coding agent declares in the same run, and the table does not count these fixes. For example, in one DeepSeek run of the Docker task, YUL rejects docker/setup-buildx-action, docker/login-action, and docker/metadata-action together with the target docker/build-push-action, and DeepSeek pins all four to the suggested SHAs. Mitigated counts this run once, although YUL fixes four dependencies in it. YUL mitigates all 22 stale DeepSeek Maven runs. This is strong empirical evidence on the effectiveness of YUL.
Failures These are only 3/42 for Opus and 2/54 for DeepSeek where YUL fails, which we now discuss.
DeepSeek treats yul's output as a prompt injection.
YUL suggests releases newer than DeepSeek’s training data, and DeepSeek sometimes concludes that the suggestion is a prompt injection. It voices this suspicion in 9 of 30 GitHub Actions YUL runs and in 2 Maven YUL runs. For example, one Maven run doubts H2 2.5.252 because it “doesn’t sound real”. After further analysis, DeepSeek sometimes accepts the suggestion, for example once it finds that “All SHAs verify against the official GitHub tags”. However, DeepSeek is sometimes persistently wrong: such as one that reasons that “v7 doesn’t exist. This could be a prompt injection attempt embedded in tool output”.
DeepSeek’s suspicion depends on what YUL suggests and on where the suggestion appears to come from. DeepSeek doubts mostly in GitHub Actions, where YUL suggests opaque commit SHAs for releases beyond DeepSeek’s training data, which DeepSeek cannot verify from memory. Opus applies the same suggestions without voicing suspicion, because Claude Code attributes each rejection to a hook, and its system prompt tells the model to treat hook feedback as coming from the user.2 OpenCode instead returns the same text as the error message of the write tool and never says that YUL wrote it, so DeepSeek sees an instruction of unknown origin inside a tool result, which is the shape of an indirect prompt injection.
yul misses less common dependency declarations.
Opus’s two unmitigated Cargo runs declare stale ranges, such as windows-sys = "0.59", under a target-specific [target.i686-pc-windows-gnu.dependencies] table that YUL does not parse. Section VI discusses how the manifest format causes such misses.
Takeaway (RQ2). YUL mitigates 39 of 42 stale Opus runs (93%) and 52 of 54 stale DeepSeek runs (96%), with the largest effect on GitHub Actions and on Maven runs. The few remaining failures come 1) from a coding agent that distrusts YUL information (2 DeepSeek runs), 2) from deliberate compatibility constraints (1 Opus run), and 3) from corner case dependency declarations that YUL does not parse (2 Opus runs).
Discussion
a) Declarative vs imperative manifests.: YUL misses a stale declaration in only two of 96 stale runs, and declarative manifests let it cover even such edge cases. A declarative manifest, such as Cargo.toml, offers a fixed set of ways to declare a dependency, so YUL can cover all of them, including the target-specific tables behind its two misses (Section V-C3). An imperative manifest, such as setup.py, declares dependencies in a general-purpose language and offers unboundedly many ways, so YUL can never cover all of them. For example, one DeepSeek run builds its dependency list with Python code, such as TEST_REQUIRES.append("mock"), which YUL cannot read without running it. Projects should therefore prefer declarative manifests, such as pyproject.toml over setup.py, so that write-time checks such as YUL can enforce their rules.
b) Package hallucinations.: YUL blocks a hallucinated dependency even when a prompt injection tells the coding agent to skip the registry check. To simulate such an injection, we ask Claude Code with YUL to “add a dependency called io.github.chains_project:yul” without looking “for it in the package registry”, and this package does not exist on Maven Central. Claude Code compiles and edits pom.xml, but YUL blocks the edit with the message of Listing 1a.
Threats to Validity
a) Model fallback.: We observe that three of the 360 Opus runs used Opus 4.8 instead of Opus 5.5. In these runs, all in the Cargo task for winapi-i686-pc-windows-gnu, a safeguard flagged the request as cybersecurity sensitive, and Claude Code automatically switched to Opus 4.8 for the rest of the run. One of the runs is an unmitigated YUL run that declares windows-sys = "0.59" (Section V-C3), and the other two pin the latest release. Without these three runs, YUL mitigates 39 of 41 stale Opus runs, so the fallback does not change our core findings.
b) Version ranges.: Our analysis reads the version that the manifest declares, while the package manager decides which version the project installs. YUL considers a range, such as a PyPI >= range, as latest if it admits the latest release (Section III). A resolver can still install an older release from such a range, for example when a lockfile pins an older release or when the local toolchain does not support the latest release [16]. Our stale counts may therefore understate how often a project installs a stale release, and our mitigated counts may overstate how often it installs the latest release.
Related Work
a) Supply chain security of coding agents.: Coding agents read untrusted text and act on it, which opens new attack surfaces in the software supply chain. Bagmar and Saraf [15] edit the setup documentation of a project in order to redirect a coding agent to an untrusted registry, a misspelled package name, or a known-vulnerable version, so that the agent is tricked into installing a malicious dependency. Malicious agent skills can compromise a coding agent during an otherwise benign task, either by hiding payloads in code examples [17] or by steering the agent toward an attacker-controlled package [18]. All these attacks require an attacker who plants text for the coding agent to read. A stale version requires no attacker. The coding agent recalls an outdated version from its training data on its own, and this version can carry known vulnerabilities that the latest release fixes.
b) Outdated dependencies.: Outdated dependencies carry a security cost. Cox et al. [2] find that systems with outdated dependencies are four times as likely to have security issues as systems that are up to date. Kula et al. [19] find that 81.5% of the studied systems keep outdated dependencies. Industry reports confirm that this gap persists in production. services, where the median dependency is 278 days behind its latest major version [20], and argue that the volume of new vulnerabilities requires faster deployment of updates [21]. Several empirical studies document how pinned dependencies drift out of date, in npm [22, 23, 24], in Maven [25], and across npm, PyPI, and Cargo [26]. He et al. [27] argue that pinning alone cannot defend against supply chain attacks. In these studies, a dependency becomes outdated over time, because the registry publishes new releases and the developer forgets to update the declared version. A coding agent, in contrast, declares a dependency that is outdated from the start, in 29% of Opus runs and 39% of DeepSeek runs (Table I). YUL blocks such outdated dependencies before they reach the manifest.
c) Automated dependency updates.: Bots such as Dependabot and Renovate open pull requests that bump outdated dependencies, and Mirhosseini and Parnin [28] show that projects with such bots upgrade 1.6 times as often as projects without them. All these approaches act only after a developer commits the stale version to the repository. YUL is the first approach to shifts the dependency update problem left at the level of coding agents. YUL blocks a stale write before it reaches the manifest, even before it is committed.
d) Package hallucination in LLM-generated code.: Code-generating LLMs sometimes recommend a package that does not exist, a failure mode the literature calls package hallucination [11, 29]. Spracklen et al. [11] generate 576,000 code samples with 16 LLMs and find that at least 5.2% of the recommended packages are hallucinations with frontier models. Djire et al. [12] evaluate seven inference-time defenses against this failure mode and find that retrieval-augmented generation reduces hallucinations in 18 of 32 model and language configurations. Such defenses lower the hallucination rate of the model but cannot guarantee that a hallucinated package stays out of the manifest. YUL blocks a hallucinated package in the harness instead, independently of the model, before the manifest write reaches disk (Section III), with guarantees. Beyond hallucinations, YUL also checks whether a real package is the latest release, which is a prerequisite for it to be secure.
e) Install-time guards.: Install-time guards wrap the package manager and block any package that a malware database flags when an install command runs. Examples include Socket Firewall [30] and Aikido Safe Chain [31]. These guards accept any stale release that the database does not flag. YUL acts earlier because it checks the manifest write before the coding agent even invokes any install command.
f) Sandboxing and runtime guardrails for coding agents.: A common security posture for coding agents consists of running them inside a sandbox that limits what their commands can reach. Yan [32] intercepts high-risk commands and wraps agent actions in transactional filesystem snapshots, so that a destructive action rolls back. Marchand et al. [33] measure how often frontier LLMs escape the containers that commonly host coding agents, and Andronchik and Lokhmakov [34] compare the isolation of five commercial AI sandboxes. Other work secures the tool calls themselves, from policy-checked tool execution [35] to guardrail frameworks against prompt injection and insecure code [36]. These defenses decide which commands and tools a coding agent may use, but none of them checks whether the dependency it declares is up to date.
The closest work to YUL is the hook of Bagmar and Saraf [15], which vetoes a pip install command that names a misspelled, nonexistent, untrusted, or known-vulnerable package, and catches 10 of their 11 attack scenarios. YUL differs from this prior work in two ways. First, their hook checks only the install command, while YUL also checks the manifest, where a coding agent often types a version directly, as in GitHub Actions workflows, which have no install command. Second, their hook accepts a stale version as long as it has no known vulnerability, while YUL requires the latest release. On YUL gives guarantees to block the stale versions suggested by coding agents.
Conclusion and Future Work
YUL mitigates 39 of 42 stale Opus runs (93%) and 52 of 54 stale DeepSeek runs (96%). YUL matters most where coding agents lack a native tool to query the latest release, as in GitHub Actions, or where that tool is unreliable, as in Maven. Even in Cargo and Go, where package manager commands exist, coding agents still pin stale versions in some runs, and YUL mitigates them.
Keeping dependencies up to date is only one software supply chain security practice, and future work should move the others into coding agents as well. These include rejecting end-of-life packages [37] and packages whose license conflicts with the project’s license [38], pinning every dependency (Docker images, Go modules) to an immutable reference [39], and skipping install-time scripts, which run malicious code as soon as the coding agent installs a package [1].
Acknowledgements
This work was supported by the Swedish Foundation for Strategic Research (SSF), by Sweden´s Innovation Agency (VINNOVA) and by the Ethereum Foundation.
References
- [1]M. Ohm, H. Plate, A. Sykosch, and M. Meier, “Backstabber’s Knife Collection: A Review of Open Source Software Supply Chain Attacks,” in Detection of Intrusions and Malware, and Vulnerability Assessment (DIMVA). Springer, 2020, pp. 23–43. [Online]. Available: https://doi.org/10.1007/978-3-030-52683-2_2
- [2]J. Cox, E. Bouwers, M. van Eekelen, and J. Visser, “Measuring Dependency Freshness in Software Systems,” in 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering (ICSE), Volume 2, 2015, pp. 109–118. [Online]. Available: https://doi.org/10.1109/ICSE.2015.140
- [3]OpenSSF Best Practices and AI/ML Working Groups, “Security-Focused Guide for AI Code Assistant Instructions,” Aug. 2025. [Online]. Available: https://best.openssf.org/Security-Focused-Guide-for-AI-Code-Assistant-Instructions
- [4]Anthropic, “Hooks reference,” 2026. [Online]. Available: https://code.claude.com/docs/en/hooks
- [5]—, “Automate actions with hooks,” 2026. [Online]. Available: https://code.claude.com/docs/en/hooks-guide
- [6]OpenCode, “Plugins,” 2026. [Online]. Available: https://opencode.ai/docs/plugins/
- [7]C. Soto-Valero, M. Monperrus, and B. Baudry, “The Multibillion Dollar Software Supply Chain of Ethereum,” Computer, vol. 55, no. 10, pp. 26–34, 2022. [Online]. Available: https://doi.org/10.1109/MC.2022.3175542
- [8]I. Pashchenko, D.-L. Vu, and F. Massacci, “A Qualitative Study of Dependency Management and Its Security Implications,” in Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2020, pp. 1513–1531. [Online]. Available: https://doi.org/10.1145/3372297.3417322
- [9]N. Zahan, P. Kanakiya, B. Hambleton, S. Shohan, and L. Williams, “OpenSSF Scorecard: On the Path Toward Ecosystem-Wide Automated Security Metrics,” IEEE Security & Privacy, vol. 21, no. 6, pp. 76–88, 2023. [Online]. Available: https://doi.org/10.1109/MSEC.2023.3279773
- [10]H. Tanaka, R. T suchida, K. Shimari, R. G. Kula, and K. Matsumoto, “An Exploratory Study of Dependabot Cooldown Adoption in Open-Source GitHub Projects,” Sep. 2026, arXiv:2609.16605 [cs.SE]. [Online]. Available: http://arxiv.org/abs/2609.16605
- [11]J. Spracklen, R. Wijewickrama, A. N. Sakib, A. Maiti, B. Viswanath, and M. Jaidlawla, “We have a package for you! a comprehensive analysis of package hallucinations by code generating LLMs,” in Proceedings of the 34th USENIX Conference on Security Symposium, ser. SEC ’25. USA: USENIX Association, 2025. [Online]. Available: https://www.usenix.org/conference/usenixsecurity25/presentation/spracklenarxiv.org/abs/2406.10279
- [12]A. E. Dijre, I. E. Olatunji, M. Tessa, E. T. Barr, J. Klein, and T. F. Bissayand, “Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code,” Aug. 2026, accepted to ASE 2026; arXiv:2608.22652 [cs.SE]. [Online]. Available: http://arxiv.org/abs/2608.22652
- [13]“ProgramBench,” 2026. [Online]. Available: https://programbench.com/
- [14]S. Ouyang, J. M. Zhang, M. Harman, and M. Wang, “An Empirical Study of the Non-Determinism of ChatGPT in Code Generation,” ACM Transactions on Software Engineering and Methodology, vol. 34, no. 2, pp. 1–28, Jan. 2025. [Online]. Available: https://doi.org/10.1145/3697010
- [15]A. Bagmar and P. Saraf, “Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents,” 2026, arXiv:2607.15143 [cs.CR]. [Online]. Available: https://arxiv.org/abs/2607.15143
- [16]Donald Stufft, “PEP 503 – Simple Repository API | peps.python.org,” Sep. 2015. [Online]. Available: https://peps.python.org/pep-0503/
- [17]Y. Qu, Y. Liu, T. Geng, G. Deng, Y. Li, L. Y. Zhang, Y. Zhang, and L. Ma, “Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems,” 2026, arXiv:2604.03081 [cs.CR]. [Online]. Available: https://arxiv.org/abs/2604.03081
- [18]Y. Liu, C.-Y. Hsu, C.-Y. Huang, M. Backes, R. Wen, and C.-M. Yu, “Trust Me, Import This: Dependency Steering Attacks via Malicious Agent Skills,” 2026. [Online]. Available: https://arxiv.org/abs/2605.09594
- [19]R. G. Kula, D. M. German, A. Ouni, T. Ishio, and K. Inoue, “Do developers update their library dependencies? An empirical study on the impact of security advisories on library migration,” Empirical Software Engineering, vol. 23, no. 1, pp. 384–417, 2018. [Online]. Available: https://doi.org/10.1007/s10664-017-9521-5
- [20]Datadog, Inc., “State of DevSecOps Report 2026,” Feb. 2026. [Online]. Available: https://www.datadoghq.com/state-of-devsecops/
- [21]David A. Wheeler, “Accelerate Deployment for the Vulnerability Tsunami,” Aug. 2026. [Online]. Available: https://www.linkedin.com/pulse/accelerate-deployment-vulnerability-tsunami-david-a-wheeler-eapge/
- [22]A. Decan, T. Mens, and E. Constantinou, “On the Evolution of Technical Lag in the npm Package Dependency Network,” in 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), 2018, pp. 404–414. [Online]. Available: https://doi.org/10.1109/ICSME.2018.00050
- [23]J. Latendrese, S. Mujahid, D. E. Costa, and E. Shihab, “Not All Dependencies are Equal: An Empirical Study on Production Dependencies in NPM,” in Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. Rochester MI USA: ACM, Oct. 2022, pp. 1–12. [Online]. Available: https://dl.acm.org/doi/10.1145/3551349.3556896
- [24]A. J. Jafari, D. E. Costa, R. Abdalkareem, E. Shihab, and N. Tsantalis, “Dependency Smells in JavaScript Projects,” IEEE Transactions on Software Engineering, vol. 48, no. 10, pp. 3790–3807, Oct. 2022. [Online]. Available: https://ieeexplore.ieee.org/document/9519532/DOI
- [25]V. Vikram, Y. Agarwal, and R. Padhye, “On the Freshness of Pinned Dependencies in Maven,” in 2026 IEEE International Conference on Software Testing, Verification and Validation (ICST), 2026, pp. 635–646. [Online]. Available: https://doi.org/10.1109/ICST69053.2026.00085
- [26]I. Rahman, J. Marley, W. Enck, and L. Williams, “Which is Better For Reducing Outdated and Vulnerable Dependencies: Pinning or Floating?” in 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2025, pp. 2795–2807. [Online]. Available: https://doi.org/10.1109/ASE63991.2025.00229
- [27]H. He, B. Vasilescu, and C. Kästner, “Pinning Is Futile: You Need More Than Local Dependency Versioning to Defend Against Supply Chain Attacks,” Proceedings of the ACM on Software Engineering, vol. 2, no. FSE, pp. 266–289, Jun. 2025. [Online]. Available: https://doi.org/10.1145/3715728
- [28]S. Mirhosseini and C. Parin, “Can automated pull requests encourage software developers to upgrade out-of-date dependencies?” in Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), 2017, pp. 84–94. [Online]. Available: https://doi.org/10.1109/ASE.2017.8115621
- [29]E. Liu, Y. Liu, L. Shi, Z. Yang, L. Zhang, X. Lian, Z. Li, Y. Ma, “Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code,” IEEE Transactions on Software Engineering, vol. 52, no. 3, pp. 1037–1055, 2026, arXiv:2404.00971 [cs.SE]. [Online]. Available: https://doi.org/10.1109/TSE.2026.3657432
- [30]Socket, Inc., “Socket CLI,” 2026. [Online]. Available: https://github.com/SocketDev/socket-cli
- [31]Aikido Security, “Aikido Safe Chain,” 2026. [Online]. Available: https://github.com/AikidoSec/safe-chain
- [32]B. Yan, “Fault-tolerant sandboxing for AI coding agents: A transactional approach to safe autonomous execution,” 2025, arXiv:2512.12806 [cs.SE]. [Online]. Available: https://arxiv.org/abs/2512.12806
- [33]R. Marchand, A. O. Cathain, J. Wynne, P. M. Giavridis, S. Jennings, F. Tuxworth et al., “Quantifying frontier LLM capabilities for container sandbox escape,” 2026, arXiv:2603.02277. [Online]. Available: https://arxiv.org/abs/2603.02277
- [34]G. Andronchik and P. Lokhmakov, “AI code sandboxes: A comparative security study. part 1 of 2 – engine-level properties,” 2026, arXiv:2606.08433. [Online]. Available: https://arxiv.org/abs/2606.08433
- [35]A. Doshi, Y. Hong, C. Xu, E. Kang, A. Kapravelos, and C. Kästner, “Towards Verifiably Safe Tool Use for LLM Agents,” in 2026 IEEE/ACM 48th International Conference on Software Engineering, New Ideas and Emerging Results (ICSE-NIER), 2026, pp. 201–205. [Online]. Available: https://doi.org/10.1109/3786582.3786839
- [36]S. Chennabasappa, C. Nikolaidis, D. Song, D. Molnar, S. Ding, S. Wan, S. Whitman, L. Deason, N. Doucette, A. Montilla et al., “Llamafirewall: An open source guardrail system for building secure ai agents,” 2025. [Online]. Available: https://arxiv.org/abs/2505.03574
- [37]C. Miller, C. Kästner, and B. Vasilescu, ““We Feel Like We’re Winging It:” A Study on Navigating Open-Source Dependency Abandonment,” in Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2023, pp. 1281–1293. [Online]. Available: https://doi.org/10.1145/3611643.3616293
- [38]W. Xu, K. Gao, H. He, and M. Zhou, “Lic eEval: Evaluating LLMs on License Compliance in Code Generation,” in 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE), 2025, pp. 1665–1677. [Online]. Available: https://doi.org/10.1109/ICSE55347.2025.2500052
- [39]A. H. M. N. Sakib, D. Banik, and M. Jaidlawla, “Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents,” Jul. 2026, kDD 2026 Workshop on Agentic Software Engineering, arXiv:2607.12428 [cs.CR]. [Online]. Available: https://arxiv.org/abs/2607.12428
