WorksComputer Science
Missing Security-Fix Backporting for GitHub Actions Workflows
Abstract
GitHub natively supports workflow automation through GitHub Actions, now the dominant CI/CD tool on the platform. Yet keeping these workflows secure remains an ongoing burden: maintainers often patch a vulnerability on the default branch but leave the same flaw on the release and maintenance branches that downstream consumers continue to build from. Carrying a known fix onto these divergent branches, called backporting, is rarely done, and this paper presents the first large-scale empirical study quantifying the resulting gap. From a public corpus of GitHub Actions workflow histories, we mine 1,903,191 workflow-modifying commits, scan each before and after with the static scanner zizmor to identify 69,865 security fixes across 24,238 projects, and audit every release branch to check whether each fix was carried over. The gap is pervasive: 224,392 of 474,490 audited release branches remain vulnerable, and deliberate backporting is rare and slow, with only 19,476 of 78,907 already-fixed branches confirmed as plausible backports at a median lag of 243 days. We thus design WorkflowBP, the first technique for automated security-fix backporting of GitHub Actions workflows. WorkflowBP takes a neuro-symbolic approach: it compiles a default-branch fix into a domain-specific representation called the Workflow Semantic Patch (WSP) and applies it to each divergent branch by matching jobs and steps based on their content rather than their position, while an LLM synthesizes the WSP for the most divergent cases the deterministic engine cannot resolve. On 12,000 real unpatched workflow files from our study, WorkflowBP produces a scanner-checked backport for 67.9%. We further submit 176 issues with WorkflowBP-generated backports to real-world projects, 44 of which have been confirmed or fixed by developers.
Introduction
In November 2019, GitHub released GitHub Actions1, a cloud-native tool that integrates Continuous Integration, Deployment, and Delivery (CI/CD) into its platform, letting developers write a workflow as a YAML file that automates tasks in response to repository events, for instance, running the test suite on every pull request or building and publishing a release artifact on a version tag. It has since become the dominant workflow automation tool on GitHub, supporting more than 3M workflows across over 49.2K public repositories [1, 2, 3]. This convenience, however, comes with security risk [4, 5]: a workflow that builds a release or deployment can silently pull in a compromised third-party component and ship an artifact that carries the flaw. This is also exemplified by several documented vulnerabilities such as CVE-2025-30066 [6], where a threat actor rewrote the release tags of the widely used tj-actions/changed-files action to point at a commit containing malicious code [4], so that every workflow referencing the action by tag executed that code on its next run, leaking CI/CD secrets across thousands of repositories.
Such a flaw is of course fixable, but a factor that is easy to overlook is how open-source projects are maintained. A project rarely lives on a single branch [7, 8]: alongside the default branch, maintainers keep long-lived release and maintenance branches so that each shipped version can receive fixes without being forced onto the latest code, and downstream users pin to and build from those branches for stability [9, 10]. Fixing only the default branch is therefore insufficient, as the vulnerable workflow persists on every release branch that was not touched. Indeed, the security analyses following the tj-actions incident stressed that patching the default branch alone leaves exposed the older branches and released versions that downstream users still build from [4, 5]. Figure 1 shows this concretely: the maintainer repins actions/checkout to an immutable commit ID on the default branch, but the release/v1.x branch still references the mutable tag @v4 and remains exposed.

Figure 1. The backporting gap in miniature. A maintainer hardens actions/checkout on the default branch (a→b) by pinning the mutable tag to an immutable commit ID, yet this fix is never backported to the release/v1.x branch (c), which still references @v4 and remains exposed to the tag-rewrite attack of CVE-2025-30066 [6].
Carrying a fix onto the branches that remain exposed is called backporting, i.e., transferring a security fix that already exists on one branch onto the divergent branches that still carry
Corresponding author: Jieke Shi. the weakness. Prior studies [9], [11, 12, 13, 14] of backporting in other settings, most notably Linux kernel security patches [9], [14], have found that developers frequently fail to propagate fixes to all the branches that need them. Whether the same gap afflicts GitHub Actions workflows, however, is unknown, and it is the question this paper sets out to answer.
Our study addresses this open question with the first investigation of how often security fixes fail to propagate across branches in open-source projects that use GitHub Actions. From the largest public corpus of workflow histories [1], collected between 2024 and 2026, we retain the 1,903,191 workflow-modifying commits across 24,238 projects, and scan each one before and after the change with ZIZMOR [15], a static analyzer for GitHub Actions. We label a commit a security fix when it removes a weakness and introduces none, yielding 69,865 fixes. Since these fixes land on the default branch, they protect only the workflows built from that branch, so we then ask whether they also reach the release and maintenance branches that downstream users continue to build from. We re-scan each such branch and, where the weakness is gone, compare when it disappeared against the timing of the default-branch fix, which lets us identify a deliberate backport, i.e., a later and separate remediation. Our results show that the gap is wide and slow to close: of 474,490 release branches, 224,392 (47.3%) remain vulnerable, and of those eventually fixed, only 19,476 received a deliberate backport, arriving at a median lag of 243 days.
A gap of this scale cannot realistically be closed by hand, which raises the question of whether workflow backporting can be automated. Existing backporting techniques, however, do not transfer to GitHub Actions. Prior work on transplantation and backporting [9], [11, 12, 13, 14], [16] mostly builds on programming-language syntax matching or even semantics, such as reasoning over and editing abstract syntax trees, driven by type and data-flow reasoning and validated against a compiler and test suite, none of which a YAML workflow, being a declarative configuration format rather than a general-purpose language, exposes. Dependency updaters such as Dependabot [17] and Renovate [18] are the only backport-like tools, yet they move a single dependency forward to a newer version and can neither couple edits across multiple locations in a workflow file nor move a fix backward onto an older branch. We therefore run the applicable options on our datasets: a heuristic copy-paste of the source diff, a Dependabot-style update, and a large language model, which has shown exceptional capability in programming [19], prompted to rewrite the file. Our empirical evaluation confirms the gap: these methods repair only 18.0%, 13.0%, and 49.5% of files respectively (), still leaving most branches unbackported.
Reading these failures exposes three challenges that any workflow backporting tool must overcome. Multiplicity: one fix is missing on a mean of 31.3 release branches per project (up to 2,971), so it must be replayed across many branches. Coupling: 34.3% of clean fixes bundle two or more weakness types into one commit whose edits are correct only together. Structural divergence: on over 60% of unpatched branches, the fix’s original location no longer exists, as release branches rename jobs, reorder steps, and refactor files. Each baseline stumbles on one of these, i.e., divergence defeats copy-paste, coupling defeats the updater, and unresolved external state defeats the LLM, and no tool addresses all three, which is the gap WorkflowBP fills.
WorkflowBP. We present WorkflowBP, the first neurosymbolic technique for automated security-fix backporting of GitHub Actions workflows. Given a security fix already applied on the default branch, WorkflowBP compiles the fixing commit into a Workflow Semantic Patch (WSP), a domain-specific representation that records the fix as a compact program of add, delete, and rewrite edits over the workflow’s structure rather than its text; applies this single WSP to each vulnerable release branch by binding its anchors to whatever the branch actually contains; and validates the result, accepting a patch only when four independent oracles agree that the targeted weakness is gone and nothing else broke. For the most divergent fixes that the deterministic engine cannot place, WorkflowBP falls back to a large language model under counterexample-guided inductive synthesis (CEGIS), which proposes a revised WSP from the engine’s counterexamples but never edits the file itself, so every accepted patch is validated by re-scanning rather than trusted.
Our WSP thus resolves all three challenges at once: it applies as a single atomic unit, so a coupled fix never lands half-applied (Coupling); its edits anchor to each job and step by content, i.e., the job name and the action in runs, rather than by position, so it still resolves when jobs are renamed and steps reordered (Structural divergence); and it fans out autonomously across a project’s hundreds of affected branches, each validated on its own (Multiplicity). On 12,000 real unpatched files from our study, WorkflowBP produces a scanner-checked backport for 67.9%, and 44 of 176 submitted as issues were confirmed or fixed by maintainers.
The main contributions of this paper are:
Study. The first large-scale study of the security-fix backporting gap in GitHub Actions workflows (24,238 projects, 474,490 branches), showing it pervasive (47.3% unpatched) and slow (median 243 days).
Benchmark. A reusable dataset of security fixes, unpatched branches, and confirmed backports mined from real histories, as ground truth for evaluating backporting techniques.
Approach. WorkflowBP, the first neuro-symbolic technique to backport a coupled workflow fix onto a divergent branch as one patch: a deterministic engine compiles the fix into a Workflow Semantic Patch and matches jobs and steps by content, an LLM resolves the most divergent cases, and every patch is validated by re-scanning.
Evaluation. We evaluate WorkflowBP on 12,000 real unpatched files against copy-paste, Dependabot-style, and LLM baselines.
Practice. We submit 176 backports as issues to real-world projects, 44 confirmed by maintainers.
Background and Motivations
GitHub Actions in Brief
GitHub Actions is a workflow automation feature built into GitHub that lets maintainers define pipelines which run in response to events such as commits and pull requests, automating tasks like testing, building, and deployment [3, 20]. Workflows are written as YAML files under .github/workflows/, and Figure 2 shows one for continuous integration. The on: field is the trigger, i.e., the events that start a run, so here every push and pull request launches the pipeline. A workflow contains one or more jobs, each running on a fresh virtual machine selected by runs-on, and each job is an ordered list of steps. A step either invokes a reusable component called an action through a uses: clause, optionally configured with with: inputs, or runs a shell command through a run: clause. The example checks out the repository, installs dependencies, builds and tests the project, and uploads a coverage report, composing the third-party actions actions/checkout and codecov/codecov-action. This composable syntax has made GitHub Actions the most widely used CI/CD service on GitHub [1, 21, 22, 23], relied on by many large and security-critical projects for continuous integration [24, 25, 26].

Figure 2. An example GitHub Actions CI workflow, run on every push and pull request. It composes two third-party actions, actions/checkout and codecov/codecov-action; each uses: clause is a point at which an external, trust-dependent component enters the pipeline.
Workflow Security Weaknesses and the Backporting Gap
The routine workflow in Figure 2 already carries three weaknesses that recur across our corpus. Unpinned actions (unpinned-uses) are referenced by a mutable tag rather than an immutable SHA, so whoever controls the tag controls what every consumer runs, the mechanism behind CVE-2025-30066 [6]. Excessive token permissions (excessive-permissions) arise when a missing permissions block lets GITHUB_TOKEN inherit repository-wide write access. Persisted credentials (artipacked) occur because actions/checkout writes the token to disk unless persist-credentials: false is set. These weaknesses are seldom fixed one at a time: a single commit usually addresses all three, and the remediation is complete only when every edit lands together, since pinning the actions while leaving credentials persisted still leaks the token. Such a fix is typically applied only to the default branch, while the same vulnerable workflow lingers on the release branches downstream users build from. Carrying it onto those branches, i.e., backporting, must reproduce the fix’s security intent in each branch’s own context rather than copy its text, since release branches diverge in action versions, runtime, and job structure. Divergence also blurs what counts as a backport, as a release branch mixes forward merges, cherry-picks, and branch-specific patches, so a present fix may reflect an incidental auto-merge rather than a deliberate patch, a distinction we make precise in §III-C.
The Backporting Gap: Empirical Studies
Although workflow security fixes are often applied to the default branch, little is known about whether these fixes are consistently propagated to the release branches that downstream users continue to rely on. We therefore conduct a large-scale empirical study of how security fixes in GitHub Actions workflows propagate from the default branch to release branches. The study is organized around three research questions and produces two ground-truth artifacts, namely unfixed branches and confirmed backports, that we later use to evaluate WorkflowBP.
RQ1 (Gap). Do security fixes on the default branch reach the release branches that downstream users build from?
RQ2 (Practice). How often, and how quickly, are release branches fixed by deliberate backporting?
RQ3 (Challenges). What dimensions of difficulty must an automated backporting tool overcome?
Data Collection
All three research questions build on a common pipeline, described next, that turns raw workflow commits into a labeled set of clean security fixes.
Commit selection. We start from a public dataset of GitHub Actions workflow histories [1] and retain entries that modify workflow files under .github/workflows/. This dataset is mined from real-world public repositories, not synthetic or toy projects: it records the full revision history of more than 116,000 workflow files across over 25,000 active public GitHub repositories [1], so the fixes we study and the release branches we audit come from projects that downstream users actually depend on. The dataset contains 3,418,911 file-level workflow changes across the corpus. We exclude entries that (i) add or delete files, for which paired pre- and post-change workflow contents are unavailable; (ii) correspond to rename or copy operations; or (iii) contain YAML that cannot be parsed or is not recognizable as a GitHub Actions workflow. After filtering, our working set contains 1,903,191 workflow-changing commits, which we use throughout the remainder of §III-A.
Before/after scanning. For each commit, we run the static workflow security scanner ZIZMOR [15] on the workflow file in both its pre-commit and post-commit state, obtaining two sets of security issues present before and after the change.
| Outcome | Branches | Share |
| Inapplicable (file absent) | 171,191 | 36.1% |
| Already-fixed | 78,907 | 16.6% |
| Still vulnerable (GAP) | 224,392 | 47.3% |
| Total release branches audited | 474,490 | 100% |
Table I. Outcome of auditing every release branch of the 24,238 projects against the corresponding default-branch fix.
| Rule | Rate | Rule | Rate |
artipacked | 93.6% | unsound-contains | 53.8% |
secrets-inherit | 92.9% | use-trusted-publishing | 53.5% |
unpinned-uses | 91.6% | bot-conditions | 45.1% |
superfluous-actions | 84.5% | github-env | 36.9% |
excessive-permissions | 81.7% | dangerous-triggers | 33.0% |
archived-uses | 79.2% | obfuscation | 13.5% |
misfeature | 73.2% | insecure-commands | 11.8% |
template-injection | 66.4% | unsound-condition | 10.3% |
unpinned-images | 62.1% | unredacted-secrets | 0.0% |
overprovisioned-secrets | 59.6% |
Table II. Per-rule gap rate on release branches, computed as over the branches whose default-branch fix touched that rule.
Security-fix filter. Let and be the sets of security issues ZIZMOR reports on the workflow before and after the commit, respectively. From them we derive the issues the commit removes, (present before, absent after), and the issues it introduces, (absent before, present after). We label a commit a security fix when it removes at least one issue and introduces none: (i) and (ii) . We adopt this two-part criterion for a specific reason. Condition (i) requires the commit to remove a real weakness, so that we study genuine fixes rather than unrelated edits. Condition (ii) requires it to introduce no new weakness, so that the change we later backport has an unambiguous security effect, i.e., a pure improvement rather than one that trades one weakness for another.
RQ1: How Wide Is the Cross-Branch Backporting Gap?
We apply the security-fix filter to all 1,903,191 commits to obtain the set of default-branch fixes whose security effect is unambiguous. For each such fix we enumerate every release-style branch of the project through the GitHub API, identifying release branches by naming convention (§VI), fetch the workflow at each branch’s current HEAD, and re-scan it, labeling the branch GAP (the fixed weakness is still present), already-fixed (absent), or inapplicable (the workflow file does not exist), then aggregate to the project level. This is a point-in-time comparison against each branch’s current HEAD: it measures whether a branch is still exposed, while how an already-fixed branch got that way (a deliberate backport or an incidental merge) is deferred to RQ2 (§III-C).
The gap is pervasive. The security-fix filter yields 69,865 default-branch fixes across 24,238 repositories (a mean of 2.9 per repository), each a weakness a maintainer demonstrably knew how to fix. Auditing every one of the 474,490 release branches these fixes could apply to (TABLE I), 224,392 (47.3%) still carry the fixed weakness, nearly three times the 78,907 already fixed, while 171,191 (36.1%) do not contain the workflow file at all. Restricted to the branches that actually carry the workflow, i.e., those on which a backport is even meaningful, the gap rises to 74.0% (224,392 of 303,299); this is the denominator behind the per-rule rates below. At the project level, 7,172 of 24,238 projects (29.6%) have at least one release branch the default-branch fix never reached. The gap is not an artifact of a few outliers: it is spread across thousands of projects, and SIII-D examines how many branches a single fix must reach.
Does the gap persist on active branches? The median date of repositories’ last workflow-modifying commits is May 2025, and 70.1% of repositories modified workflows in 2025. We randomly sample 600 release branches and down-weight older branches by halving their weight for each six-month period since their last commit. Among vulnerable branches, 53.7% received commits within the past 24 months. The weighted proportion of vulnerable branches is 49.2%. The rate is similar when we consider only branches updated within the 6, 12, or 24 months before our audit in June 2026 (46.5%, 47.5%, and 50.1%).
Which weaknesses are left behind. Normalizing the gap per rule, defined as over the branches whose default-branch fix touched that rule, exposes which weaknesses fare worst (TABLE II). The supply-chain rules dominate: artipacked (93.6%), secrets-inherit (92.9%), and unpinned-uses (91.6%) are left unpatched on the large majority of branches that once carried them, the very weaknesses behind the tj-actions class of attacks (§III-B).
Is the gap real? Validating the oracle. The entire gap measurement rests on ZIZMOR’s verdicts, so we validate it along two axes. For reproducibility, we pin one version (1.24.1) and scan content-addressed YAML blobs, making every verdict deterministic; ZIZMOR’s rules encode the official hardening guidance of GitHub [27], the OpenSSF Scorecard [28], and documented CVEs [6], covering the weaknesses of §II-B and further supply-chain and configuration categories. For accuracy, two authors independently adjudicated every ZIZMOR finding on, and checked for weaknesses it missed in, a uniform random sample of 200 workflow files: ZIZMOR achieves 96.8% precision and 99.1% recall. In the security-fix filter, a false positive that disappears after a commit makes the commit count as a security fix, even though it removed no real weakness. With this precision, such cases are a small minority.
Finding 1. The cross-branch backporting gap is pervasive: 47.3% of audited release branches still carry a weakness already fixed on the default branch (74.0% of the branches that carry the workflow at all), and 29.6% of projects have at least one such branch. The supply-chain rules behind recent action-tag attacks are left unpatched on over 90% of the branches that once carried them.
RQ2: Characterizing Backporting Practice
For each of the 78,907 already-fixed branches, we walk the workflow file’s commit history on that branch to locate the commit where the issue first disappeared, and compare its date with the default-branch fix. We classify each branch as a plausible backport (the branch had the issue and removed it more than a day after the default branch), same-day (removed on the same day, most likely an auto-merge), independent prior fix (removed before the default branch), never had the issue, or inconclusive when the history is too deep to resolve within a bounded per-record budget, i.e., at most 50 commits per file and 16 minutes per check. For confirmed plausible backports, we compute the lag from the default-branch fix.
Results. Walking the history of the 78,907 already-fixed branches (Table III), 19,476 (24.7%) are confirmed plausible backports, distributed across 1,279 projects. A clear majority of the already-fixed branches are same-day removals (41,408, 52.5%), most likely reflecting an auto-merge from the default branch rather than a deliberate decision to patch the release line. The confirmed backports are slow: the median lag is 243 days, with 1,576 landing within one to three months, 9,065 within three to twelve, and 6,580 taking more than a year, up to a maximum of 2,326 days. Only 1,296 land within a single week, driven by a few security-conscious projects such as demergent-labs/azle (466 of its 5,581 confirmed backports). For most release branches, the window of exposure to an already-fixed weakness is measured in months.
| Refined category | Branches | Share |
| Plausible backport (lag day) | 19,476 | 24.7% |
| Same-day (likely auto-merge) | 41,408 | 52.5% |
| Independent prior fix (lag day) | 5,670 | 7.2% |
| Never had the issue | 4,020 | 5.1% |
| Inconclusive (history too deep) | 4,582 | 5.8% |
| Timed out (budget exceeded) | 3,751 | 4.8% |
Table III. Refined classification of the 78,907 ALREADY-FIXED branches by file-history analysis.
Finding 2. Deliberate backports are rare, concentrated, and slow: 19,476 of 78,907 already-fixed branches (24.7%) are confirmed plausible backports; the median lag is 243 days, and although a security-conscious minority lands 1,296 within a single week, the long tail extends past six years.
RQ3: Deriving Backporting Challenges
The approaches available today. Before turning to what makes backporting hard, we measure how far a maintainer would get with the approaches available today, run on a common set of 12,000 unpatched files (Table IV): a verbatim copy-paste of the source diff; a Dependabot-style single-dependency update, which advances one action to a newer version but cannot add the coupled permission or credential edits that travel with a hardening fix; and a general-purpose LLM prompted to rewrite the whole file. These close only 18.0%, 13.0%, and 49.5% (Gemini) or 33.7% (GPT) of the files, respectively, so each leaves most branches unfixed, and each fails for a different reason. We now name the three dimensions of difficulty the data reveals and tie each to the approach it defeats.
| Approach | Passed | Rate |
| Pure-LLM rewrite (Gemini) | 5,940 | 49.5% |
| Pure-LLM rewrite (GPT) | 4,044 | 33.7% |
| Copy-paste of source diff | 2,158 | 18.0% |
| Dependabot-style update | 1,565 | 13.0% |
Table IV. What the approaches available today achieve on the common set of 12,000 unpatched files. Pure-LLM counts only patches whose action pins resolve to real commits.
Multiplicity (One Fix Often Spans Many Branches)
1) Multiplicity (One Fix Often Spans Many Branches): Backporting is challenging because a single default-branch fix may need to be propagated to many release branches. Among the 7,172 projects with at least one unpatched release branch, the mean number of gap branches is 31.3 per repository. Moreover, 752 distinct default-branch fixes are each missing from more than 50 release branches, including datalens-tech/datalens-ui (2,971 branches), datalens-tech/datalens-backend (555), demergent-labs/azle (2,759), n8n-io/n8n (434), and azure/azure-sdk-for-go (434). These results show that workflow backporting is often a one-to-many problem, where a project may not be fully remediated until the same fix is adapted and applied across tens or even hundreds of release branches.
Implication for tool design. Closing this gap by hand would mean opening up to 2,971 pull requests for a single fix and re-solving conflicts on each, which is why it is often left undone. A tool must instead fan one extracted fix out across all of a project’s affected branches autonomously, validating each on its own.
Coupling (One Commit, Multiple Coordinated Edits)
2) Coupling (One Commit, Multiple Coordinated Edits): 34.3% of security fixes (23,965 of 69,865) couple two or more distinct ZIZMOR rule categories in a single commit, and the {artipacked, excessive-permissions, unpinned-uses} triple alone accounts for 13.4% (9,380) of all security fixes. This shows that real remediations are commit-level atomic units, rather than single-change patches.
Implication for tool design. A coupled fix is correct only as a whole, since pinning the action but leaving persist-credentials set still leaks the token. A per-dependency updater cannot express this: a Dependabot-style update raises a single action’s version but never adds the coupled permissions and persist-credentials edits, nor moves an already-vetted fix backward onto an older branch, and it closes only 13.0% of the unpatched files (Table IV). A tool must therefore treat the commit as its atomic unit, accepting a backport only if every edit lands, never a partial one.
Structural Divergence (Release Branches Do Not Match the Default Branch in Shape)
3) Structural Divergence (Release Branches Do Not Match the Default Branch in Shape): Release-branch YAML structures diverge from default-branch structures on most projects: job names are renamed, step orderings differ, and files are refactored. In 60.5% of the unpatched branches in our dataset, the location the fix occupied on the default branch no longer exists on the release branch, so the fix cannot be applied where it originally sat. Figure 3 contrasts a default-branch fix with a structurally divergent release branch that carries the same weakness.

Figure 3. Why a textual diff cannot be backported verbatim. The default-branch patch targets jobs.build.steps[0].with.persist-credentials, but on the release branch the job is renamed (prepare), an extra setup step shifts the checkout to index 1, and no with block exists yet.
Implication for tool design. Without the fix’s literal path, a verbatim copy-paste of the source diff has no pre-image to match and cannot apply; even where a hunk does apply, it reuses the source branch’s pinned SHA rather than the version the target runs, which can downgrade or break the branch. Copy-paste closes only 18.0% of the unpatched files (TABLE IV). A tool must instead locate the target by structural content rather than position, binding job and step to whatever the branch actually contains, and express each edit as a desired end-state that can create missing structure, rather than as a textual hunk. Even a general-purpose LLM prompted to rewrite the whole file, which handles the structural rewriting the other two approaches cannot, closes only 49.5% of the unpatched files (TABLE IV). No single existing approach handles multiplicity, coupling, and structural divergence together, and this is the gap WORKFLOWBP closes.
Finding 3. Workflow backporting in practice exhibits three operational dimensions of difficulty: multiplicity, coupling, and structural divergence. The three dimensions are orthogonal: each must be addressed for a backporting tool to be useful at scale.
WorkflowBP
WORKFLOWBP takes as input a security fix that has already been applied on the default branch and propagates it to the release branches that still contain the vulnerable workflow. As Figure 4 shows, it operates in four steps.
Figure 4. Overview of WORKFLOWBP. A default-branch fix is compiled once into a Workflow Semantic Patch (WSP), applied to each divergent release branch by content-based matching, and checked by four independent oracles. Cases where the deterministic engine declines, or that fail validation, are routed to a counterexample-guided LLM that synthesizes a revised WSP, which is then held to the same apply-and-verify path.
Step 1 (Compile). WORKFLOWBP turns the fixing commit into a Workflow Semantic Patch (WSP), which is a program
in a small domain-specific language (DSL) that describes the edits the fix makes to the workflow’s structure rather than to its text (§IV-B).
Step 2 (Apply). It replays this single WSP on each release branch that still carries the vulnerability, binding the WSP’s anchors to whatever the branch actually contains (§IV-C).
Step 3 (Validate). It accepts a patch only when four independent oracles agree that the targeted vulnerability is gone and nothing else broke (§IV-D).
Step 4 (LLM fallback). For the cases the deterministic engine declines or that fail validation, WORKFLOWBP asks a large language model to synthesize a revised WSP from the engine’s counterexamples, holding the result to the same oracles (§IV-E).
The same WSP that the engine executes is also what a reviewer reads, since it is a short, human-readable program, so every backport is auditable before it becomes a pull request. The rest of this section defines the WSP language and then details each step.
The Workflow Semantic Patch (WSP)
The WSP is a small domain-specific language (DSL) that records a fix as edits to the workflow’s tree structure rather than to its text. Its design is inspired by Cocci n elle’s Semantic Patch Language (SmPL) [29], a tool that expresses code changes as structural patterns rather than line diffs, from which it borrows the @@-delimited header, the +/- edit lines, and metavariables, i.e., named placeholders that match a piece of structure the patch then edits. It differs in three ways that the workflow-backporting setting demands: a WSP is compiled automatically from one before/after diff rather than hand-written, its edits are idempotent assertions about a desired end state rather than imperative hunks, and it targets YAML and GitHub Actions semantics, namely jobs, steps, and uses identity, rather than a C abstract syntax tree.
Figure 5 gives the core grammar (top) and a compiled example (bottom). A program opens with a header that names the master source, lists the ZIZMOR rules the commit removed, and declares the job metavariables it uses; the body is a sequence of blocks, each an anchor that points into the workflow tree followed by the edits that should hold there.

Figure 5. The WSP language: its core grammar (top, EBNF, simplified) and the compiled program that backports a fix (bottom). The header names the master source, lists the ZIZMOR rules removed, and declares one job metavariable per touched job. Each block is an anchor (a path of mapping keys, a $J job metavariable, and content [field=value] selectors) followed by edits: a lone + is ENSURE_PRESENT, a lone - is ENSURE_ABSENT, a -/+ pair on the same key is a rewrite, and <sha: pin ...> resolves the target branch’s own reference at apply time. The full grammar is in our artifact.
Anchors. An anchor is a path built from three kinds of segment. A key segment descends into a mapping by its literal name, such as permissions or steps. A selector, written [uses=actions/checkout], picks one element out of a list by matching its content rather than its index, here the checkout step inside a job’s step list, so the anchor still resolves after the branch has reordered or inserted entries. Steps can also be selected by [id=...] or [name=...]. A job metavariable, written $J and declared in the header, stands for one job the master fix touched. Its declaration carries its identity: pin “build” is the literal job key the master edited, recover uses=actions/checkout names a fingerprint step, i.e., a step that no sibling job runs, used to relocate the job if the branch renamed it, since a unique step lets the engine identify the job by what it does rather than by its name, and bind one requires it to bind exactly one job, stopping and deferring to human review rather than fanning the edit out to every job. A multi-job fix declares one metavariable per touched job (J_2,...), each with its own edits. We anchor at the job level because renamed jobs are the most common form of structural divergence across release branches.
Edits. Each edit specifies the desired state of one key, and the operation is derived from the signs in its block. A lone + key: v is ENSURE_PRESENT (the key must be present with value v); a lone - key is ENSURE_ABSENT (the key must be absent); and a - key: old paired with a + key: new on the same key is a REWRITE. Edits are idempotent assertions, not hunks: an assertion that already holds is a no-op, so replaying a WSP onto a partially fixed or drifted branch converges instead of double-applying. One value form, the reference to an external action, is special. A WSP encodes the security intent behind such a reference rather than the default branch’s concrete SHA. The fix on the default branch replaces a mutable tag, e.g., @v4, with the exact commit that tag pointed to, which removes the risk that the tag is later rewritten to malicious code. Copying that same commit onto a release branch can be unsafe, however, since the default branch may run a newer action version than the release branch, so the copied commit would silently upgrade the branch. A uses pin is therefore written as the marker <sha: pin target_ref>, which tells the engine to read the reference the target branch itself uses and pin it to the exact commit it currently points to. This keeps the branch on its own action version while removing the mutable tag, so the security intent, i.e., no rewritable tag, is preserved without changing which version runs.
Master Commit to WSP
WorkflowBP compiles the master commit, together with the rules the fix removed, into a WSP. The compiler works structurally, not textually. It first flattens the before and after workflows into maps from a structural path to a leaf value, where a path keys each list element by its content identity, for instance jobs.build.steps[uses=actions/checkout].with.persist-credentials, rather than by its index. Comparing the two maps by set algebra yields the leaves the commit added, removed, and changed. The remaining stages turn these leaves into edits.
Deriving job metavariables. The compiler declares one metavariable for each job touched by the diff, using pin “job” bind one to anchor it to the literal job key and recover to specify a fingerprint: a step whose uses action is unique to that job, since a job’s name may change on a release branch but the action it runs usually does not. A job with no globally unique step gets no fingerprint and fails closed on rename rather than risk the wrong job. Keeping each touched job on its own metavariable matters when a commit edits several jobs differently, since a shared anchor would fan one job’s edit out to the others, widening privileges on jobs the fix never touched.
Consolidating type changes. Most leaves become one edit each, but a value that flips between a scalar and a block does not. For example, secrets: inherit (a scalar) becoming a secrets: mapping with several entries (a block) would decompose into an ENSURE_ABSENT on secrets and several ENSURE_PRESENT children, where the ENSURE_ABSENT would delete the whole secrets key before its children could be written. The compiler detects such scalar-to-block transitions and emits a single rewrite that replaces the old value with the entire new block at once.
Declining whole-step changes. An edit can set fields inside an existing step, but it cannot create a step that is not there. When the master fix adds or removes an entire step, the compiler cannot know where that step belongs on a branch whose job has a different set of steps in a different order. Rather than guess, it flags the edit for review and routes the case to the LLM fallback described below; validation later refuses any WSP whose edits did not all land.
Filtering to the security intent. A mined security-fix commit is rarely surgical: alongside the security change it often bumps a run script, a with argument, or a matrix entry, and replaying those onto a release branch would revert content the branch evolved independently. The compiler therefore auto-applies only edits attributable to a targeted rule’s security construct, through a fixed rule-to-construct map (e.g., excessive-permissions → permissions, unpinned-uses → uses, template-injection → {env, run}), where a construct is the specific workflow key a rule concerns. Every other edit is flagged for review and never lands, so a backport carries the fix’s security content and nothing else.
WSP to Release Branch
WORKFLOWBP applies a WSP by resolving each block’s anchor against the release branch’s parse tree and then making the block’s edits. Resolution walks the anchor segment by segment: a key descends by name; the job metavariable binds the pinned job, or, if it was renamed, the one job matching the recover fingerprint, with an ambiguous match held for review and the engine never inventing a job; and a [uses=...] selector matches a step by content rather than index. Because every segment resolves against the branch’s own structure, one compiled WSP applies across branches whose jobs and steps have moved.
A resolved anchor may stop short when only mapping keys are missing, for instance a job with no permissions block yet; the engine creates such keys for an ENSURE_PRESENT edit but never invents a missing job or step. The edits then run in their derived form: ENSURE_PRESENT sets a key, ENSURE_ABSENT deletes one, and REWRITE replaces a value. A uses pin is resolved here: the engine reads the reference the target step carries, queries GitHub for the commit it points to, and writes that SHA, adding the original tag as a trailing comment for readability; a reference that does not resolve becomes a review item, never a guess. All edits use a format-preserving YAML round-trip, so comments, quoting, and indentation survive.
Validation
A backport is acceptable only if it removes the targeted weakness without introducing a new one, leaves the workflow runnable, and changes nothing else. WORKFLOWBP checks this with four oracles that are independent of the WSP engine, and accepts a patch only if all four agree. (1) Security. WORKFLOWBP re-runs ZIZMOR, the static analyzer used throughout to detect workflow weaknesses, on the target before and after the patch, requiring every edited scope to end free of the targeted findings with no new finding inside it. The check is scoped to the edited regions because a branch may carry other, pre-existing instances of the same rule that the fix never addressed. (2) Validity. ACTIONLINT, a linter that checks GitHub Actions workflow syntax and semantics, must report no new finding relative to the target, the strongest static proxy for “the workflow still runs”, since actually executing the workflow would need the project’s secrets and CI environment. (3) Permission containment. No untouched job’s own permissions block may change, catching the over-grant and over-strip that a scanner misses, since any explicit permissions block silences excessive-permissions regardless of what it grants. (4) Minimality. A structural diff of the target and patched file must attribute every changed leaf to a targeted rule’s construct. This oracle uses no scanner, so it does not simply re-confirm the scanner-based definition of a fix, and it catches a patch that silences the scanner while reverting the branch’s own run or with content.
Recovering Declined Cases with LLM Assistance
On its own, the deterministic engine cannot handle every fix. It declines a fix that needs a whole step added or removed, because it cannot tell where the step belongs on a branch whose steps differ in number and order, for instance a fix that inserts a hardening step before checkout when the target job has a different step sequence. It also declines a fix whose anchor cannot be bound, for instance when the branch renamed the touched job and no longer runs a step that uniquely identifies it, so the job metavariable matches nothing under bind. These are the most divergent cases, where a static rewriter would have to guess. To recover some of them, WORKFLOWBP turns to an LLM, while holding it to exactly the same oracles it applies to its own patches.
We frame this as counterexample-guided inductive synthesis (CEGIS) 2 over the WSP language. The model never edits the workflow; it proposes a WSP, and the engine stays the only component that touches the file, so format preservation, idempotent anchoring, and deterministic pinning still hold. Given the master fix’s before and after files, the security-relevant part of its diff, the touched jobs, the target file, and the rules to remove, the model emits a candidate WSP, which WORKFLOWBP applies and runs through the four oracles. On rejection it returns concrete counterexamples, namely anchors that did not resolve, the edits it could not place, and each oracle violation, and the model revises. The loop runs for a fixed number of rounds (), and a case that never passes is reported as unsolved. Because every candidate is machine-applied and machine-checked, the fallback inherits the same discipline as the engine: a WSP that does not apply and pass the checks is never accepted.
Evaluation
We evaluate WORKFLOWBP on the unpatched workflow files from our study, asking two questions:
RQ4 (Comparison). How does WORKFLOWBP compare with the approaches a maintainer could use today?
RQ5 (Cost). What are the cost of WORKFLOWBP compared with general-purpose LLMs?
Beyond these two questions, we execute 17 of the accepted backports (§V-E) and submit 176 backports as issues to real-world projects (§V-F).
Setup
The evaluation runs on the unpatched workflow files from our study. Each pair of a default-branch security fix and a release branch still carrying the corresponding weakness yields one target file. The 224,392 unpatched release branches yield 312,401 evaluation cases, as multiple default-branch fixes may apply to the same branch. Running every baseline on all 312,401 cases is not affordable, since two baselines are commercial LLMs invoked with a multi-round repair loop that can cost more than 4,000 USD. We therefore stratify the fixes into three groups by the degree of workflow restructuring, ranging from fixes that only modify existing workflow elements to those that add or remove entire steps, and randomly sample 12,000 cases in proportion to each group’s size. For each method we report the count and fraction of these 12,000 files for which it produces an accepted backport, accepted only when the four validation oracles agree the targeted weakness is gone and nothing else broke.
Implementation Details
The LLM fallback and the pure-LLM baselines use Gemini-3.1-flash-lite and GPT-5.4-mini (model identifiers google/gemini-3.1-flash-lite and openai/gpt-5.4-mini), accessed in June 2026. All requests set temperature 0 and a limit of 8,192 output tokens, and leave top-p at its default. In each round, the pure-LLM baselines output a rewrite of the whole file. In contrast, WORKFLOWBP’s fallback generates and revises a WSP, which the symbolic engine applies to the target file. Both approaches allow up to three rounds, with the previous output and validation feedback provided after each unsuccessful attempt. We use ZIZMOR 1.24.1 and ACTIONLINT 1.7.12.
RQ4: Comparison with Baselines
RQ4 sets WORKFLOWBP against the three approaches available to a maintainer today: a verbatim copy-paste of the source diff, a Dependabot-style [17] single-dependency updater, and a general-purpose LLM that rewrites the whole file, whose failure modes RQ3 tied to the three challenges. To keep the LLM comparison fair, the pure-LLM baseline uses the same model as the WORKFLOWBP configuration it is compared against, so we compare WORKFLOWBP (Gemini) with pure-LLM (Gemini) and WORKFLOWBP (GPT) with pure-LLM (GPT); the only difference is whether the model rewrites the whole file or only synthesizes a Workflow Semantic Patch (WSP), which the engine then applies and pins to the branch’s live action versions. All methods run on the same 12,000 files, and for the pure-LLM baseline we count a patch only when its action pins resolve to real commits, since an unresolvable pin would not run and so would not help.
| Method | Passed | Rate |
| WORKFLOWBP (Gemini-3.1-flash-lite) | 8,148 | 67.9% |
| WORKFLOWBP (GPT-5.4-mini) | 7,174 | 59.8% |
| Pure-LLM (Gemini) | 5,940 | 49.5% |
| Pure-LLM (GPT) | 4,044 | 33.7% |
| Symbolic engine only | 3,291 | 27.4% |
| ZIZMOR autofix | 2,664 | 22.2% |
| Copy-paste | 2,158 | 18.0% |
| Dependabot-style update | 1,565 | 13.0% |
Table V. RQ4 comparison of WORKFLOWBP against the approaches available today, on the common set of 12,000 unpatched files. Pure-LLM counts only patches whose action pins resolve to real commits.
TABLE V reports the result. WORKFLOWBP accepts 67.9% of files with Gemini and 59.8% with GPT, well above every baseline: 49.5% (Gemini) and 33.7% (GPT) for the same two models used alone, 18.0% for copy-paste, and 13.0% for Dependabot. Without the LLM fallback, the symbolic engine accepts 27.4% of the files, already above both naive baselines, copy-paste and Dependabot. The LLM fallback is invoked only on the 8,709 files the engine declines and adds the remaining 40.5% (4,857 backports) of the 67.9% with Gemini. The engine accepts 68.6% of fixes requiring no workflow restructuring but only 1.1% of fixes that add or remove entire steps, while the fallback raises this rate to 56.3%. On these 8,709 files, the fallback accepts 55.8% with Gemini and 44.6% with GPT, versus 39.7% and 18.4% for the pure-LLM baselines. Because the model only synthesizes a WSP whose pins the engine resolves against the live branch, its patches never fail on a fabricated commit hash.
We also compare with ZIZMOR’s own autofix mode (zizmor --fix=all), which closes 22.2% of the same 12,000 files under the four oracles (TABLE V), against 67.9% for WORKFLOWBP. In 21.4% of the files, ZIZMOR’s output removes none of the targeted findings, introduces a new finding, or fails ACTIONLINT, and in another 56.4% it is rejected by the minimality oracle because it introduces unrelated changes. ZIZMOR also has no autofix for excessive-permissions, secrets-inherit, or unpinned-images. Of the 4,338 files whose fix targets at least one of these rules, WORKFLOWBP’s accepted patches remove findings of these rules in 1,743 (40.2%) with Gemini and 1,583 (36.5%) with GPT, and the symbolic engine alone does so in 1,061 (24.5%). This is because WORKFLOWBP transfers the maintainer’s own remediation, which can capture project-specific requirements. For example, the Gemini patches that fix excessive-permissions grant pull-requests: write in 288 files and contents: write in 193 files. In almost all of these files, the permissions blocks are identical to those in the maintainer’s fix.
| Path | Cases | In tok. | Out tok. |
| Fallback WSP (GPT) | 8,709 | 9,066 | 446 |
| Fallback WSP (Gemini) | 8,709 | 9,032 | 501 |
| Pure-LLM rewrite (GPT) | 12,000 | 3,954 | 1,158 |
| Pure-LLM rewrite (Gemini) | 12,000 | 4,551 | 1,343 |
Table VI. Per-case LLM cost, averaged over the cases that invoke each path. Fallback rounds are capped at 3 (GPT mean 2.01, Gemini mean 1.82).
On the same 12,000 files, WORKFLOWBP differs significantly from every baseline under exact McNemar tests with Holm correction (). Over the pure-LLM baseline with the same model, WORKFLOWBP accepts 18.4 percentage points more files with Gemini (95% CI [17.1, 19.7]) and 26.1 more with GPT (95% CI [25.0, 27.2]).
Finding 4. WORKFLOWBP accepts up to 67.9% of unpatched files, about 3.8 times the best naive baseline (copy-paste at 18.0%) and above ZIZMOR’s autofix (22.2%) and the pure-LLM baseline (49.5%) under the same model.
RQ5: Cost
WORKFLOWBP concentrates its cost on the cases that need it. The symbolic engine is deterministic, performing a structural diff and two ZIZMOR scans, before and after the edit, per branch with no LLM API call, so the files it accepts on its own and the large fan-outs the multiplicity challenge requires, i.e., replaying one fix across a project’s many branches (e.g., 2,971 branches for datalens-ui), cost zero tokens and parallelize freely. The LLM fallback fires only on the 8,709 of 12,000 sampled files that the engine cannot place, capped at three rounds (mean 2.01 under GPT, 1.82 under Gemini). Because the model emits a compact WSP (about 450 to 500 output tokens) rather than a full-file rewrite (1,200 to 1,340), and output tokens dominate API cost (TABLE VI), the entire evaluation runs at well under one US cent per accepted backport.
Finding 5. The deterministic engine accepts 27.4% of the files at zero LLM API cost, including fan-outs of thousands of branches, and the fallback runs only on the files the engine cannot place, at a few hundred output tokens per case, keeping the per-backport cost well below one cent.
Execution Checks
We cannot execute all patched workflows because doing so would require access to each project’s secrets, runners, and triggering events. Due to limited time, we randomly selected and executed 17 workflows that could be completed within the available time, and all 17 WORKFLOWBP-patched versions completed successfully. By design, the symbolic engine does not generate new code; it transfers maintainer-authored edits from the default branch, reducing the risk of unintended behavior.
Maintainer Responses
We selected non-archived projects, prioritizing those whose vulnerable release branches were most recently updated, and submitted 176 issues, each including WORKFLOWBP’s generated patch. Of these, 44 have been confirmed; 16 have been remediated, including 2 merged patches and 14 affected branches removed, with another 8 fixes under review. The merged backports, covering 6 workflow files, provide real-world evidence of patch correctness, as they match WORKFLOWBP’s patches and execute successfully in the maintainers’ environments. Among the remaining 132 submissions, 62 were declined because maintainers had no plans to maintain the branch (34), considered the reported weakness not applicable to their use case or branch (22), or did not accept reports generated by automated tools (6). The remaining 70 have not yet received a response. Because most remediations removed the affected branch and the largest group of declined issues concerns branches the maintainers had no plans to maintain, WORKFLOWBP is most useful on the release branches that a project still maintains.
Threats to Validity
Internal validity. Both the study and WORKFLOWBP rely on ZIZMOR, a static analyzer for GitHub Actions, to identify security fixes and to decide whether a backport removes the targeted weakness, so our results inherit its accuracy. Manual validation on a random sample of 200 workflow files showed that ZIZMOR achieves 96.8% precision and 99.1% recall. We applied the same validation to ACTIONLINT, the linter WORKFLOWBP uses to confirm a patched workflow remains syntactically valid, obtaining 97.9% precision and 99.4% recall.
External validity. Our corpus comes from public GitHub Actions histories and skews toward active, well-maintained projects, so behavior on dormant repositories may differ. We identify release branches from naming conventions (e.g., release/* or version-tagged names), so projects that name their maintenance branches differently are missed and their gaps go uncounted. What’s more, our evaluation runs on a sample of 12,000 of the 312,401 cases (§V-A). To assess representativeness, we further ran WORKFLOWBP with the lower-cost DeepSeek-V4-Flash on all 312,401 cases. It accepts 66.5% of the full population versus 67.3% on the sample, indicating that the sample closely reflects the full population.
Construct validity. We count a release branch as backported only when the fix lands more than a day after the default-branch fix, excluding same-day removals, which we cannot distinguish from automatic merges, so our backport counts are a lower bound. Acceptance is a static proxy for a correct and runnable patch: ZIZMOR confirms the targeted weakness is gone with nothing new flagged, and ACTIONLINT checks that the patched workflow remains syntactically valid, standing in for execution. We executed 17 randomly selected accepted backports that could be completed within the available time, and all 17 completed successfully (§V-E); the merged backports, covering 6 workflow files, also execute successfully in the maintainers’ environments (§V-F). We do not run the other patched workflows themselves, which would require each project’s secrets and CI/CD infrastructure, so a patch that passes our oracles may still fail at runtime.
Related Work
Empirical studies of GitHub Actions. Prior work characterizes Actions adoption and operation: Decan et al. [3] find 43.9% of nearly 70K repositories use Actions, Golzadeh et al. [2] link its rise to the decline of Travis, CircleCI, and Azure, and Kinsman et al. [20] study activity shifts after adoption. Saroar and Nayebi [32] survey practitioner perceptions, Valenzuela-Toledo and Bergel [33] study workflow maintenance, and Zheng et al. [24] taxonomize 375 failed runs across 260 projects. None of these ask whether a default-branch fix reaches the release branches downstream users build from, which is the gap our study quantifies.
Workflow and CI/CD security. Static analyzers such as ZIZMOR [15], GHAST [34], ARGUS [35], and those surveyed by Fares et al. [36] detect weaknesses within a single, isolated workflow file. Large-scale audits confirm such weaknesses are pervasive [25], [34], [37], [38], sitting within a broader supply-chain risk mapped by studies of malicious packages [39], the npm threat surface [40], and attack taxonomies [41], and made concrete by incidents such as CVE-2025-30066 [6]. This line of work reports that a branch is vulnerable but does not transfer a fix across branches, which is the gap WORKFLOWBP closes.
Dependency updating. Automated updaters such as Dependabot [17] and Renovate [18] raise one pull request per dependency, moving forward toward the latest release. Empirical work shows the limits of this model: vulnerabilities propagate widely through the npm dependency network [42], fixes reach downstream consumers only after long lags [43], an ecosystem-level analogue of the 243-day median backport lag we observe, and automated pull requests often overwhelm developers [44]. This single-dependency, forward-only model cannot express a coupled hardening fix, nor move a vetted fix back onto a divergent release branch, and it closes the smallest share of the gap among the methods we evaluate.
Patch backporting and transplantation. Transferring a fix across versions or programs is a decade-old line of work for source code: LASE applies a learned edit across many locations [16], muScalpel and CodeCarbonCopy transplant functionality between programs [45], [46], PatchWeave transplants a patch into a semantically similar program [13], and FixMorph and TSBPORT backport Linux kernel patches [9], [14]. Studies of backporting practice confirm the task is hard and error-prone [47]–[50]. These techniques rely on programming-language semantics, ASTs, types, control- and data-flow, and a compiler or test suite, none of which a YAML workflow provides. Nor do they face the problem workflow backporting requires: resolving external state by pinning an action to the commit its tag currently points to. Meanwhile, related work evaluates SZZ implementations for tracing bug-fixing commits to bug-inducing commits [51], while we examine how fixes propagate across release branches.
LLM-based repair. Where YAML semantics are thin, LLM-based repair, such as zero-shot LLMs that rewrite vulnerable code [52], [53] or LLM-based agents that drive automated program repair [54], [55], remains relevant, and Hou et al. [19] survey their broad programming capability. Empirical studies further examine how practitioners build and evaluate SE agents [56] and what users expect from AI coding assistants [57]. However, LLMs cannot recover mutable external state, such as an action’s live commit SHA, and instead hallucinate it. WORKFLOWBP uses a model only to synthesize a WSP and leaves SHA resolution to the deterministic engine, so no accepted patch rests on a fabricated SHA.
Conclusion
We presented the first large-scale study of security-fix backporting in GitHub Actions workflows. Across 1,903,191 commits we identified 69,865 security fixes in 24,238 repositories, and found the resulting backporting gap pervasive and slow: 47.3% of audited release branches remain vulnerable, 29.6% of projects are affected, and where a fix is carried over, the median lag is 243 days, with deliberate backports rare and concentrated in a small number of security-conscious projects. We therefore built WORKFLOWBP, a neuro-symbolic backporting tool: a deterministic engine compiles a fix into a Workflow Semantic Patch, a structural representation of the edits, and applies it across branches under scanner validation, while an LLM fallback, held to the same scanner validation and never trusted to edit the file directly, recovers the structurally hardest cases. WORKFLOWBP produces a scanner-checked backport for up to 67.9% of unpatched files, about 3.8× the best naive baseline and well above a pure LLM rewrite. We release the dataset and tool to support further work on workflow-level supply-chain remediation. Looking ahead, we plan to adapt WORKFLOWBP to other configuration-as-code formats such as GitLab CI and strengthen the validity signal beyond what scanners and linters provide.
Data Availability
The replication package is available at: https://github.com/CharlieMCY/WorkflowFix.
Acknowledgments
This research is supported by the National Research Foundation, Singapore, and the Cyber Security Agency of Singapore under its National Cybersecurity R&D Programme (Proposed ID: NCR25-DeSCemT-SMU). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the National Research Foundation, Singapore, and the Cyber Security Agency of Singapore.
References
References
- [1]G. Cardoen, T. Mens, and A. Decan, “A dataset of GitHub Actions workflow histories,” in Proceedings of the 21st International Conference on Mining Software Repositories, ser. MSR ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 677–681.DOI
- [2]On the rise and fall of CI services on GitHub,” in 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), 2022, pp. 662–672.
- [3]“On the Use of GitHub Actions in Software Development Repositories,” in 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME), 2022, pp. 235–245.DOI
- [4]M. Bar, S. Berkowich, and G. Nagli, “GitHub Action tj-actions/changed-files supply chain attack: everything you need to know,” 2025, accessed: 2026-06-03. [Online]. Available: https://www.wiz.io/blog/github-action-tj-actions-changed-files-supply-chain-attack-cve-2025-30066
- [5]Cycode Team, “GitHub Actions supply chain attacks: How they work and how to stop them,” 2026, accessed: 2026-06-24. [Online]. Available: https://cycode.com/blog/github-actions-supply-chain-attack/
- [6]MITRE, “CVE-2025-30066,” 2025, accessed: 2026-06-03. [Online]. Available: https://nvd.nist.gov/vuln/detail/cve-2025-30066
- [7]W. Zou, W. Zhang, X. Xia, R. Holmes, and Z. Chen, “Branch Use in Practice: A Large-Scale Empirical Study of 2,923 Projects on GitHub,” in 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS), 2019, pp. 306–317.
- [8]P. Bunyakiati and U. Sammapun, “Open-Source Projects and their Collaborative Development Workflows,” 2020. [Online]. Available: https://arxiv.org/abs/1909.00642
- [9]R. Shariffdeen, X. Gao, G. J. Duck, S. H. Tan, J. Lawall, and A. Roychoudhury, “Automated patch backporting in Linux (experience paper),” in Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2021. New York, NY, USA: Association for Computing Machinery, 2021, p. 633–645.
- [10]J. Dietrich, D. Pearce, J. Stringer, A. Tahir, and K. Blincoe, “Dependency Versioning in the Wild,” in 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), 2019, pp. 349–359.DOI
- [11]E. T. Barr, M. Harman, J. Jia, A. Marginean, and J. Petke, “Automated software transplantation,” in Proceedings of the 2015 International Symposium on Software Testing and Analysis, ser. ISSTA 2015. New York, NY, USA: Association for Computing Machinery, 2015, p. 257–269.DOI
- [12]D. Yang, X. Mao, L. Chen, X. Xu, Y. Lei, D. Lo, and J. He, “TransplantFix: Graph Differencing-based Code Transplantation for Automated Program Repair,” in Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’22. New York, NY, USA: Association for Computing Machinery, 2023.DOI
- [13]R. S. Shariffdeen, S. H. Tan, M. Gao, and A. Roychoudhury, “Automated Patch Transplantation,” ACM Trans. Softw. Eng. Methodol., vol. 30, no. 1, Dec. 2021.DOI
- [14]S. Yang, Y. Xiao, Z. Xu, C. Sun, C. Ji, and Y. Zhang, “Enhancing OSS Patch Backporting with Semantics,” in Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’23. New York, NY, USA: Association for Computing Machinery, 2023, p. 2366–2380.DOI
- [15]W. Woodruff, “Welcome to zizmor’s documentation!” 2024, accessed: 2026-06-03. [Online]. Available: https://docs.zizmor.sh/
- [16]N. Meng, M. Kim, and K. S. McKinley, “Lase: Locating and applying systematic edits by learning from examples,” in 2013 35th International Conference on Software Engineering (ICSE), 2013, pp. 502–511.
- [17]GitHub, “Securing your supply chain - Enable and manage features for supply chain security,” 2026, accessed: 2026-06-03. [Online]. Available: https://docs.github.com/en/code-security/how-tos/secure-your-supply-chain
- [18]renovatebot.com, “Mend Renovate CLI documentation,” 2026, accessed: 2026-06-03. [Online]. Available: https://docs.renovatebot.com/
- [19]X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Xi, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large Language Models for Software Engineering: A Systematic Literature Review,” ACM Trans. Softw. Eng. Methodol., vol. 33, no. 8, Dec. 2024.arxiv.org/abs/2308.10620
- [20]T. Kinsman, M. Wessel, M. A. Gerosa, and C. Treude, “How Do Software Developers Use GitHub Actions to Automate Their Workflows?” in 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), 2021, pp. 420–431.DOI
- [21]P. Rostami Mazrae, A. Decan, T. Mens, and M. Wessel, “An empirical study of GitHub actions workflows,” Journal of Systems and Software, vol. 236, p. 112824, 2026.
- [22]I. Bouzenia and M. Pradel, “Resource Usage and Optimization Opportunities in Workflows of GitHub Actions,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ser. ICSE ’24. New York, NY, USA: Association for Computing Machinery, 2024.DOI
- [23]T. A. Ghaleb, O. Abduljalil, and S. Hassan, “CI/CD Configuration Practices in Open Source Android Apps: An Empirical Study,” ACM Trans. Softw. Eng. Methodol., vol. 35, no. 2, Jan. 2026.arxiv.org/abs/2411.06077
- [24]L. Zheng, S. Li, X. Huang, J. Huang, B. Lin, J. Chen, and J. Xuan, “Why Do GitHub Actions Workflows Fail? An Empirical Study,” ACM Trans. Softw. Eng. Methodol., vol. 35, no. 5, Apr. 2026.DOI
- [25]I. Koishybayeva, A. Nahapetyan, R. Zachariah, S. Muralee, B. Reaves, A. Kapravelos, and A. Machiry, “Characterizing the Security of Github CI Workflows,” in 31st USENIX Security Symposium (USENIX Security 22). Boston, MA: USENIX Association, Aug. 2022, pp. 2747–2763.
- [26]M. Moazen, A. M. Ahmadian, and M. Balliu, “Granite: Granular Runtime Enforcement for GitHub Actions Permissions,” 2025. [Online]. Available: https://arxiv.org/abs/2512.11602
- [27]GitHub, “Secure use reference - Security practices for writing workflows and using GitHub Actions features,” 2026, accessed: 2026-06-18. [Online]. Available: https://docs.github.com/en/actions/reference/security/secure-use
- [28]N. Zahan, P. Kanakiy, B. Hambleton, S. Shohan, and L. Williams, “OpenSSF Scorecard: On the Path Toward Ecosystem-Wide Automated Security Metrics,” IEEE Security & Privacy, vol. 21, no. 6, pp. 76–88, 2023.DOI
- [29]J. Lawall and G. Muller, “Coccinelle: 10 Years of Automated Evolution in the Linux Kernel,” in 2018 USENIX Annual Technical Conference (USENIX ATC 18). Boston, MA: USENIX Association, Jul. 2018, pp. 601–614.
- [32]S. G. Saroar and M. Nayebi, “Developers’ Perception of GitHub Actions: A Survey Analysis,” in Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering, ser. EASE ’23. New York, NY, USA: Association for Computing Machinery, 2023, p. 121–130.arxiv.org/abs/2303.04084
- [33]P. Valenzuela-Toledo and A. Bergel, “Evolution of GitHub Action Workflows,” in 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), 2022, pp. 123–127.
- [34]G. Benedetti, L. Verdiame, and A. Merlo, “Automatic Security Assessment of GitHub Actions Workflows,” in Proceedings of the 2022 ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses, ser. SCORED’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 37–45.DOI
- [35]S. Muralee, I. Koishybayeva, A. Nahapetyan, G. Tystahl, B. Reaves, A. Bianchi, W. Enck, A. Kapravelos, and A. Machiry, “ARGUS: A Framework for Staged Static Taint Analysis of GitHub Workflows and Actions,” in 32nd USENIX Security Symposium (USENIX Security 23). Anaheim, CA: USENIX Association, Aug. 2023, pp. 6983–7000.
- [36]M. Fares, Y. Gamage, and B. Baudry, “Unpacking Security Scanners for GitHub Actions Workflows,” 2026. [Online]. Available: https://arxiv.org/abs/2601.14455
- [37]H. Onsori, O. Delizieh, A. Decan, and T. Mens, “Quantifying Security Issues in Reusable JavaScript Actions in GitHub Workflows,” in Proceedings of the 21st International Conference on Mining Software Repositories, ser. MSR ’24. New York, NY, USA: Association for Computing Machinery, 2024, p. 692–703.
- [38]Z. Pan, W. Shen, X. Huang, Y. Yang, R. Chang, Y. Liu, C. Liu, Y. Liu, and K. Ren, “Ambush From All Sides: Understanding Security Threats in Open-Source GitHub CI/CD Pipelines,” IEEE Transactions on Dependable and Secure Computing, vol. 21, no. 1, pp. 403–418, 2024.DOI
- [39]M. Ohm, H. Plate, A. Sykosch, and M. Meier, “Backstabber’s Knife Collection: A Review of Open Source Software Supply Chain Attacks,” in Detection of Intrusions and Malware, and Vulnerability Assessment, C. Maurice, L. Bilge, G. Stringhini, and N. Neves, Eds. Cham: Springer International Publishing, 2020, pp. 23–43.
- [40]M. Zimmermann, C.-A. Staicu, C. Tenny, and M. Pradel, “Small World with High Risks: A Study of Security Threats in the npm Ecosystem,” in 28th USENIX Security Symposium (USENIX Security 19). Santa Clara, CA: USENIX Association, Aug. 2019, pp. 995–1010.arxiv.org/abs/1902.09217
- [41]P. Ladisa, H. Plate, M. Martinez, and O. Barais, “SoK: Taxonomy of Attacks on Open-Source Software Supply Chains,” in 2023 IEEE Symposium on Security and Privacy (SP), 2023, pp. 1509–1526.DOI
- [42]A. Decan, T. Mens, and E. Constantinou, “On the impact of security vulnerabilities in the npm package dependency network,” in Proceedings of the 15th International Conference on Mining Software Repositories, ser. MSR ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 181–191.DOI
- [43]B. Chinthanet, R. G. Kula, S. McIntosh, T. Ishio, A. Ihara, and K. Matsumoto, “Lags in the release, adoption, and propagation of npm vulnerability fixes,” Empirical Softw. Engg., vol. 26, no. 3, May 2021.
- [44]S. Mirhseinli and C. Parnin, “Can automated pull requests encourage software developers to upgrade out-of-date dependencies?” in 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), 2017, pp. 84–94.
- [45]A. Marginean, E. T. Barr, M. Harman, and Y. Jia, “Automated Transplantation of Call Graph and Layout Features into Kate,” in Search-Based Software Engineering, M. Barros and Y. Labiche, Eds. Cham: Springer International Publishing, 2015, pp. 262–268.DOI
- [46]S. Sidirglou-Douskos, E. Lahtinen, A. Eden, F. Long, and M. Rinard, “CodeCarbonCopy,” in Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ser. ESEC/FSE 2017. New York, NY, USA: Association for Computing Machinery, 2017, p. 95–105.
- [47]F. Thung, X.-B. D. Le, D. Lo, and J. Lawall, “Recommending Code Changes for Automatic Backporting of Linux Device Drivers,” in 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME), 2016, pp. 222–232.DOI
- [50]J. Tasnim, D. Chakraborti, C. K. Roy, and K. A. Schneider, “How does quality deviate in stable releases by backporting?” in 2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), 2023, pp. 140–145.DOI
- [51]Y. Lyu, H. J. Kang, R. Widaryasari, J. Lawall, and D. Lo, “Evaluating SZZ Implementations: An Empirical Study on the Linux Kernel,” IEEE Transactions on Software Engineering, vol. 50, no. 9, pp. 2219–2239, 2024.arxiv.org/abs/2308.05060
- [52]H. Pearce, B. Tan, B. Ahmad, R. Karri, and B. Dolan-Gavitt, “Examining Zero-Shot Vulnerability Repair with Large Language Models,” in 2023 IEEE Symposium on Security and Privacy (SP), 2023, pp. 2339–2356.
- [53]C. S. Xia, Y. Wei, and L. Zhang, “Automated Program Repair in the Era of Large Pre-trained Language Models,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), 2023, pp. 1482–1494.DOI
- [54]Y. Zhang, H. Ruan, Z. Fan, and A. Roychoudhury, “AutoCodeRover: Autonomous Program Improvement,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2024. New York, NY, USA: Association for Computing Machinery, 2024, p. 1592–1604.arxiv.org/abs/2404.05427
- [55]H. Ruan, Y. Zhang, and A. Roychoudhury, “SpecRover: Code Intent Extraction via LLMs,” in 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE), 2025, pp. 963–974.arxiv.org/abs/2408.02232
- [56]Y. Lyu, D. Wang, J. Shi, Z. Sun, C. Peng, Z. Yang, F. Sarro, and D. Lo, “How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study,” 2026. [Online]. Available: https://arxiv.org/abs/2607.10856
- [57]Y. Lyu, Z. Yang, J. Shi, J. Chang, Y. Liu, and D. Lo, “‘My productivity is boosted, but . . .’ Demystifying Users’ Perception on AI Coding Assistants,” in 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2025, pp. 191–203.arxiv.org/abs/2508.12285
