WorksComputer Science
Query Processing with eBPF-Offload on Computational Storage
Abstract
The idea of pushing computation to storage devices has been explored for decades. In recent years, research has focused on computational storage, i.e., devices that allow a host to execute functions as I/O commands on local data. Today, the NVMe and SNIA computational storage standards share an execution model in which the host invokes functions on the device, on data that the host has moved into device memory. Both standards single out eBPF as a format for functions offloaded by the host. In this paper, we study whether this form of computational storage is a good match for query processing. We present the design and implementation of Delilah, a prototype based on an open-hardware device that implements the standard execution model on an actual device and runs eBPF programs calling device-resident functions. As an initial step, we hand-coded a portion of SSB Q3.3 for execution on the device, aiming to integrate computational storage with a database system's storage manager. The series of experiments that we ran showed that offloading gains little over host execution, execution degenerates into column-at-a-time processing, and time is spent staging and moving data in device memory. Because these roadblocks stem from the execution model rather than from the query, we did not pursue integration further. We detail these limitations and lay the groundwork for a different computational storage model better aligned with data-intensive workloads.
A. Krause, J. Pietrzyk, D. Habich, W. Lehner TU Dresden E-mail: {alexander.krause, johannes.pietrzyk, dirk.habich, wolfgang.lehner}@tu-dresden.de
N. Hedam IT University of Copenhagen E-mail: [email protected]
S. Lee, A. Lerner Computing Flows E-mail: {sangjin, lernera}@computingflows.com
P. Bonnet University of Copenhagen E-mail: [email protected] ORCID: 0000-0002-0234-5374
Introduction
To avoid the data-access bottleneck [36], data-centric architectures place processing capability near where the data resides. For data management, this idea goes back to active disks [42] and has resurfaced with each generation of storage hardware [1]. Its current incarnation is computational storage, a form of near-data processing that enables hosts to invoke functions on device data via I/O commands [29]. After a first generation of prototypes and products based on proprietary interfaces, computational storage is now standardized through SNIA1 [49, 48] and NVMe2 [38, 39].
The SNIA programming model is broad [49]. In its direct usage model, the host invokes a function on data in device memory. In its indirect usage model, the storage controller moves data from storage into device memory and triggers a function as a side effect of a storage request. Aggregator functions invoke sequences of functions on the device, possibly in parallel and possibly repeatedly, so as to operate on data with minimal host involvement.
The SNIA and NVMe interfaces that realize this model are much narrower [48, 38]. They are both explicitly tailored for a host-orchestrated interface. In NVMe, a computational program operates only on data in the device memory, denoted subsystem local memory [39]. The host copies data into that memory and executes the program with separate commands. The SNIA API [48] softens this model with batch requests, which chain storage, copy and compute requests into an acyclic graph that the device executes without interacting with the application. A batch is fixed when it is submitted, so it cannot contain loops, and its structure cannot depend on the data it processes. In short, computational storage, as exposed through standard interfaces, is based on host-invoked functions and host-managed data movement. The device performs the computation; the host orchestrates it.
This model is familiar. It is an accelerator-offload model in which a host launches kernels over buffers it explicitly populates. It is also attractive from a standardization standpoint: it adds a few operations to the storage interface while leaving the storage stack semantics untouched. The question we address in this paper is whether it is a good match for query processing.
There are reasons to doubt. First, computational storage relies on computation that accesses SSDs through their external interface [29]. As a result, computational storage has no bandwidth or latency advantage over the host CPU when accessing stored data [28]. Since computational storage increases the cost of the storage subsystem, it must pay off in other ways: by reducing the amount of data transferred to the host, by freeing host cores and memory, or by executing hardware-accelerated functions [9, 26].
Second, device memory is small and shared by data staged from the SSDs, intermediate results and results to be transferred to the host: memory, not compute, is the scarce resource on the device. Query processing stresses exactly these constraints. We cannot assume that base tables fit in device memory, so columns must be processed in chunks. Operators maintain state across chunks (e.g., selection vectors), and the amount of data flowing between them depends on selectivities that are only known at run-time. Previous work characterized offloadable functions as small, I/O-bound compute kernels that account for a significant portion of an application’s execution time [53]. Whether query operators, orchestrated from the host, fit this characterization is an open question.
Both SNIA and NVMe single out extended Berkeley Packet Filter (eBPF) as a favoured format for representing functions downloaded at run-time: SNIA lists eBPF among its execution environments, and its API defines eBPF programs as a type of downloadable function [48]; NVMe proposes eBPF as the representation of downloadable programs [38]. eBPF originates in the BSD Packet Filter, designed to evaluate boolean-valued functions without copying packets inside a UNIX kernel [33]. Modern eBPF [47] executes user-defined functions inside the Linux kernel for performance monitoring [14], security monitoring [11] and storage management [56], and serves as an intermediate representation for hardware-accelerated packet processing [5]. It would be tempting to judge eBPF for computational storage by the standards of packet filtering, as a sandboxed language in which stateless functions of a few instructions are executed on data streaming through a device, or to compare it with other bytecodes as a general-purpose data processing language [24]. We argue that this is not its role in computational storage.
In computational storage, eBPF is best understood as a relocatable binary format. eBPF is a vendor-neutral instruction set [19], generated from C by both gcc and clang. An eBPF program refers to external functions by name; the program resolves these references when it is linked against the functions registered with the run-time environment on the device. Device-defined functions, possibly hardware accelerated, can thus be registered with the eBPF run-time environment and called from a program generated on the host, without extending the host-device interface with a new operation for each function (§ 3.1). In this role, eBPF programs act as glue: they massage the inputs and outputs of registered functions and orchestrate their execution. At the same time, native or hardware-accelerated code performs most of the processing. eBPF’s limitations as a programming language (no floating point, no indirect calls, and the difficulty of verifying programs with loops) matter less in this role. However, safely executing downloaded code remains an open problem.
In SNIA terms, an eBPF program that calls registered functions is an aggregator function: it runs in an eBPF environment and invokes functions that may run in other environments, e.g., in hardware. Unlike a batch request, it can contain loops and make data-dependent decisions. eBPF does not change the execution model. Through standard interfaces, an eBPF program is a host-invoked function, executed on data the host has moved into device memory, and the sequence it orchestrates is bounded by what the host has staged. eBPF determines what to offload and how to package it; the execution model determines how offloaded functions interact with data. It is the latter that we put to the test.
To this end, we design and implement Delilah, a computational storage system that follows the host-orchestrated execution model of standard interfaces on an actual open hardware device, the OpenSSD Daisy platform [51], using an on-path PCIe architecture. Delilah runs eBPF programs in user space on the device’s ARM cores, links them against registered functions, and exchanges programs, arguments, and results with the host through program and data slots in device memory. We use Delilah to evaluate the mechanisms on which the execution model rests: the cost of invoking an offloaded function, the coherence of the buffers shared between host and device, and the transfer of data between them. We then map query processing onto this execution model: we split columns into chunks that fit in device memory, register operators as functions invoked chunk by chunk from the host, and keep two chunks in flight to overlap processing and transfers.
As a first step, we hand-code query Q3.3 from the Star Schema Benchmark (SSB). This is a sanity check before integrating Delilah into a database system’s storage manager. We chose Q3.3 because it is a selective query over a few columns with simple operators, so it should be well suited for offloading on computational storage. On this query, we run a campaign of experiments that vary what is offloaded, how filters are implemented (Neon SIMD or FPGA), and how operators are scheduled, and we compare against host-only execution.
This first step was also our last. We identified showstoppers that no amount of query tuning can work around, because they stem from the execution model itself: functions are invoked by the host, and data staging and data movement are both managed by the host. These showstoppers already appear in the most favorable setting we could construct, on a data set whose columns fit entirely in device memory. Larger data sets, more columns, and more complex plans would only multiply the per-chunk round trips and staging passes that already dominate execution time. We therefore report these negative findings rather than pursue the integration further.
Query processing calls for an execution model in which the device manages the iteration over data and the movement of data between SSD and device memory. The SNIA programming model anticipates device-side orchestration with indirect usage and variable-sequence aggregator functions, but the standard interfaces do not yet expose them. However, we argue that device-side orchestration is not enough. We propose a more holistic approach to SSD programming that addresses computational storage’s shortcomings.
In summary, our contributions are the following:
We survey computational storage standards and clarify eBPF’s role in computational storage.
We describe the design and implementation of Delilah, a computational storage system following the host-orchestrated execution model on a real device (Section 3), and a chunked, column-at-a-time execution model for offloaded query processing (Section 4).
We evaluate eBPF offload overhead and host-device data transfers on Delilah. Through a campaign of experiments on a hand-coded SSB query, we analyze where time goes when offloading (Section 5). We show that the limitations we encounter are structural, and conclude that host-invoked functions and host-managed data movement are not a good match for query processing.
We draw the lessons from this work and discuss perspectives for near-data processing with co-designed SSDs.
This article extends our DaMoN’23 paper [19], which introduced Delilah. The new material provides a detailed analysis of the revised SNIA and NVMe standards, details of the Delilah system including the controller, driver and block design, and the offload of query processing, including the chunked execution model and its evaluation with an SSB query. Code and experiments are available online at https://github.com/delilah-csp.
Related Work
Computational storage [2] is the current incarnation of decades-old ideas about active storage [42] and near-data processing [1]. It refers to architectures where storage functions are executed, as a result of I/Os, on compute resources located outside Solid-State Drives. Computational storage functions thus only have access to the external interface of SSDs. This distinguishes computational storage from programmable and co-designed SSDs, where programs have access to internal SSD interfaces [29].
We organize related work along two dimensions: execution model and code representation. The execution model determines who invokes offloaded functions, and who moves the data they operate on. Code representation concerns how offloaded functions are represented. Here, we focus on eBPF.
Execution Model
Proprietary interfaces. The first generation of commercial computational storage used proprietary interfaces, such as ScaleFlux [6], NGD [20], Samsung SmartSSD [45], or Seagate Kinetic Drive [35]. In particular, Cao et al. [6] tackle analytical workloads in the cloud-native relational database POLARDB with ScaleFlux computational storage. Each device defined its own interface, its own set of offloadable functions, and its own division of labor between host and device—the SNIA and NVME standardization efforts aimed at reducing this diversity.
SNIA defines a programming model for computational storage, together with an application programming interface; NVMe defines the device interface. We review them in turn.
SNIA programming model. The SNIA Computational Storage Architecture and Programming Model [49], first released in 2022 and approved as version 1.2 in April 2026, defines the terminology, architecture and programming model of computational storage. It makes no assumption about the physical interface between host and device. It distinguishes three kinds of computational storage devices (CSx): computational storage processors (CSP), which provide computation for an associated storage system without persistent storage; computational storage drives (CSD), which also contain persistent storage; and computational storage arrays (CSA). The computational storage resources of a CSx comprise:
computational storage engines (CSE), i.e., components such as a CPU or an FPGA that execute computational storage functions (CSF);
computational storage engine environments (CSEE), i.e., the operating environment of a CSE, e.g., an operating system, a container platform, an eBPF environment or an FPGA bitstream;
a resource repository that holds CSFs and CSEEs available for activation;
function data memory (FDM), portions of which are allocated (AFDM) to individual instances of CSF operations. An instance may only access its own device memory.
CSEEs and CSFs are either pre-installed or downloaded by the host into the resource repository.
The programming model defines two usage models. In the direct usage model, the host sends a request that invokes a CSF. In the reference flow, the host places the input data in AFDM before the request, and the CSE places its results in device memory; the model also allows inputs and outputs to reside in device storage or in host memory.
In the indirect usage model, the host configures the storage controller to associate a CSF with storage requests, based on request parameters, on an address range or on data characteristics. The storage controller then moves data from storage into device memory, triggers the CSF, and returns its results in response to the storage request.
Version 1.1 added sequencing, to operate on data with minimal host involvement. An aggregator CSF invokes a sequence of activated CSFs on the same device, sequentially or in parallel, possibly across CSEEs. A fixed-sequence aggregator hard-codes the sequence. A variable-sequence aggregator determines it from parameters and intermediate results, possibly invoking the same CSF repeatedly.
One example of the standard splits a large input into chunks that parallel search CSFs process before combining their results. The standard also lists example functions that are directly relevant to query processing, including a database filter CSF (projection and selection) and a pipeline CSF that combines commands according to a data flow specification. The SNIA programming model is thus broader than host-invoked functions and host-managed data movement: indirect usage delegates data movement to the storage controller, and aggregator CSFs move the orchestration of functions onto the device.
SNIA API. The SNIA Computational Storage API [48], a SNIA standard whose version 1.1 was released in May 2025, defines the interface between applications and computational storage devices. It is agnostic to how devices are attached (directly, over a network or over a fabric): a library, extended with plugins, maps API functions onto the specific interface of each device, e.g., NVMe. A plugin may also substitute host CPU execution for a device that is absent. Besides discovery and configuration functions, which activate CSEEs and CSFs, the API defines functions to download CSFs (including eBPF programs, one of the CSF types of the API) and four groups of functions that structure the use of a device:
memory management functions to allocate and free device memory;
storage requests load data from storage into device memory, or store data from device memory to storage, at the block or file level, possibly with peer-to-peer transfers that bypass host memory;
copy requests move data between host memory and device memory, or within the device;
compute requests execute an activated CSF, whose arguments are scalars or references to device memory.
All requests are queued and complete synchronously or asynchronously (via a callback or an event). The reference flow follows the direct usage model of the programming model: the application allocates device memory for inputs and outputs, queues a storage request to load data, queues a compute request, and queues a copy request to retrieve the results. The API is explicit about the resulting division of labor. It is tailored for a host-orchestrated interface. Even when data is transferred peer-to-peer between SSD and AFDM, the host remains in charge of orchestrating it. Correspondingly, the API defines no function that associates a CSF with storage requests, as the indirect usage model would require.
The API mitigates the cost of host orchestration with batch requests. A batch combines storage, copy and compute requests, executed serially, in parallel, or as a graph of dependencies (hybrid mode), and the device executes it without interacting with the application. Batches may be supported in hardware or emulated in software. A batch is, however, fixed when it is submitted: an entry that would create a loop is rejected, the number of entries is bounded by the device, and the structure of the batch cannot depend on the data it processes. Batch requests thus implement a form of fixed-sequence aggregation, defined by the host, but not the variable-sequence aggregation of the programming model.
NVMe. NVMe defines the device interface for computational storage with two command sets, Computational Programs (TP 4091) and Subsystem Local Memory (TP 4131), now ratified [38, 39]. They correspond to CSFs and FDM in the SNIA model. Computational programs operate only on data in subsystem local memory (SLM). The host loads or activates a program on a computational programs namespace, describes the ranges of SLM a program operates on as a memory range set, and executes the program with a command that references this memory range set. Data is moved between NVM namespaces and SLM with copy commands, and between SLM and the host with memory read and write commands, all issued by the host. The structure is the same as with the SNIA API: data is read from an SSD into SLM by the host prior to program execution, and results are read (or written to an SSD) by the host after program execution. NVM Express itself describes its Computational Programs command set as a host-driven approach. NVMe distinguishes between two kinds of programs. Device-defined programs are installed in hardware (FPGA bitstream) or software (OS image) before a namespace is attached. Downloadable programs, represented in a hardware-agnostic bytecode such as eBPF, are loaded on a computational programs namespace by the host at run-time.
Host-invoked functions and host-managed data movement. The two standard interfaces thus agree, at the application level and at the device level: functions are invoked by the host on data that the host has moved into device memory, and results are moved by the host out of device memory. This is the execution model that we refer to as host-invoked functions and host-managed data movement. It is the model that standard computational storage interfaces expose today, and the one we evaluate in this paper. Designing and evaluating a device that supports the elements of the SNIA programming model that go beyond it, indirect usage (which can be seen as a form of database kernel [46]) and variable-sequence aggregation, remains a topic for future work.
Prototypes. Samsung demonstrated a second generation SmartSSD [44] supporting eBPF offload over NVMe at SC’22. Among systems described publicly, ZCSD [31] offloads eBPF bytecode to a computational storage device over ZNS SSDs emulated in QEMU, via a local function call. In a real system, this mechanism must be replaced by a protocol between host and device over PCIe or fabric. The Eid-Hermes project [40], led by Eideticon, defines such a protocol for eBPF offload over PCIe. It enables a host to load an eBPF program to a PCIe device, run the program synchronously and then unload it. The eBPF program is executed in user-space with the uBPF virtual machine3. Parameters are written by the host to a buffer on the device, and results are explicitly read back by the host: Eid-Hermes is a direct instance of host-invoked functions and host-managed data movement. It is implemented in QEMU. We use it as a point of departure for Delilah, which implements this execution model on an actual computational storage device (Section 3). The host interface of Delilah covers, in a simplified form, the memory, copy and compute requests of the SNIA API. The key difference is that Delilah supports registered functions that read data from the underlying SSD into the device memory.
Wilcox et al. [53] consider an architecture for code offload on computational storage, also emulated in QEMU, in which compute kernels are loaded and then executed asynchronously. Compute kernels can call a restricted library of pre-defined SSD access functions, i.e., they can initiate I/O from the device, which departs from purely host-managed data movement. Wilcox et al. also characterize the code worth offloading: a compute kernel is a small section of code which is both I/O bound and comprises a significant portion of the overall execution time of the application. A compute kernel must be small enough to be efficiently executed on the device, I/O bound to benefit from offloading to storage, and significant enough for its offload to have an impact on performance. Our evaluation of query processing (Section 5.4) shows that this characterization is not sufficient: individual operators may well be small, I/O bound and significant, yet the way they are orchestrated from the host determines where time goes.
Query processing near storage. Offloading query processing to storage has been studied extensively on programmable SSDs, where code runs on the SSD controller and reads flash through internal interfaces, i.e., on co-designed SSDs in the terminology of Lerner et al. [29]. Do et al. [8] push selection and aggregation from Microsoft SQL Server into a Samsung Smart SSD. The operators are compiled into the SSD firmware and exposed as commands; the host opens a session, the device processes the data as it is read from flash, and the host retrieves the results. Joins are not offloaded, and the data layout had to be converted to obtain speedups. Biscuit [15] generalizes this approach into a framework for a commodity NVMe SSD. Its programming model is based on data flow: tasks that run on the SSD (SSDlets, written in C++ and loaded at run-time) or on the host communicate through typed ports. The host loads a module, instantiates SSDlets, connects their ports into a graph and starts the application; SSDlets then read the files passed to them by the host through the internal I/O path, and data streams through the ports. Communication between host and device is expensive (hundreds of microseconds per port transfer), so the framework favors long-running streaming tasks over fine-grained invocations. YourSQL [25] ports MariaDB onto Biscuit and offloads early filtering. The query planner selects a table with restrictive predicates, estimates its page-level selectivity with a sampling task on the SSD, and places the table first in the join order. Filter tasks on the SSD then scan the entire table without interruption, using hardware pattern matchers on the flash channels and a software filter, and accumulate per-page match hints. The host pulls these hints, reads the qualifying pages in bulk and executes the rest of the query, in a pipeline that overlaps filtering on the device with processing on the host. In a micro-benchmark on a column-store layout of TPC-H tables, YourSQL also observed that page-level early filtering is ineffective on an uncompressed column store, where almost every page contains a qualifying value.
In all three systems, the device, not the host, drives the iteration over stored data: the host intervenes once per scan or per query, not once per chunk. Biscuit, in particular, demonstrates on a commercial NVMe SSD the kind of execution model that the SNIA programming model anticipates with variable-sequence aggregation and device-managed data movement, i.e., a host-defined graph of device-side tasks that move data themselves. It does so through a proprietary interface. Two differences with computational storage must be kept in mind. First, these systems benefit from the internal bandwidth of the SSD, which exceeds the bandwidth of its host interface: Do et al. relied on a gap of almost with SATA/SAS SSDs, and Biscuit reports roughly 1 GiB/s of additional internal bandwidth over PCIe Gen34. A computational storage device accesses SSDs through their external interface and has no such advantage. Second, they are tied to a specific SSD and its firmware. Our question is thus different: Can query processing benefit from computational storage as exposed by the standard interfaces, with host-invoked functions and host-managed data movement over the external SSD interface.
Data-driven offload. Outside computational storage standards, several lines of work offload code with an execution model in which the arrival of data or requests, rather than the host, drives execution. XRP [56] executes eBPF storage functions in the NVMe driver of the Linux kernel. A function can resubmit I/Os from the driver, e.g., to traverse an on-disk index, which avoids a round trip through the storage stack for each step of the traversal. Code offload on smart NICs has also matured rapidly. Smart NICs, i.e., network interfaces that allow stateful computation on messages or flows, play a central role in the convergence of hyperscale data center and high-performance computing networks [22]. A prime example is sPIN [21], an architecture supporting the execution of user-defined functions within a smart NIC. sPIN distinguishes between header, payload and completion handlers that are invoked as the corresponding parts of a message stream through the NIC. sPIN was recently used for offloading storage policies, including erasure coding, for disaggregated storage over RDMA [7]. In this context, throughput is non-negotiable (e.g., 200 or 400 Gb/sec): the execution of offloaded code cannot interfere with the expected NIC throughput, which restricts user-defined functions to a few instructions per packet, typically focused on data movement [43].
DPUs and SSDs. Data Processing Units (DPUs), i.e., SoC-based smart NICs with general-purpose cores, memory and accelerators, are increasingly attached directly to SSDs. In SmartNIC JBOFs, a smart NIC replaces the server in front of a set of NVMe SSDs. Gimbal [34] implements a software storage switch on such a smart NIC that schedules I/O between network ports and SSDs for co-located tenants. LEED [16] runs a persistent key-value store on an array of SmartNIC JBOFs, and Scalio [50] scales a DPU-based JBOF key-value store by serving reads through RDMA and NVMe-oF target offload rather than through the DPU cores. In database systems, DDS [55] uses a DPU in front of the SSDs of a disaggregated storage server: client read requests are executed directly on the DPU, which accesses the SSDs without involving the host, and an offload engine executes user-defined requests on the DPU, while updates remain on the host because DPU memory is small. DPDPU [23] generalizes this approach into a platform with compute, network and storage engines that expose asynchronous interfaces to data systems. Thostrup et al. [52] characterize BlueField DPUs from the perspective of a DBMS on fast networks.
These systems share two traits with our work. First, execution on the DPU is driven by requests arriving from the network, and the DPU, not the host, moves data between SSDs, its own memory and the network. They thus expose new storage interfaces (a key-value store, a page server, a file service), as anticipated by Do et al. [10] for on-path architectures, rather than the standardized computational storage interfaces. Second, they run into the same constraint as Delilah: memory on the device is the scarce resource. DDS keeps updates on the host for that reason, and Scalio has to keep DPU memory and SSD state consistent in software, since they are not kept coherent by hardware.
Giouroukis et al. [13] study how DPU accelerators perform on query-related tasks (compression, decompression and regular expression filtering) applied to data arriving from the network. In their setting, the host decides which tasks to offload and submits them one by one. They find that DMA-intensive offloads compete with the NIC for the DPU’s memory interconnect, and that the best placement between host and DPU varies with the operator, the data and the query. Above all, they find that this fine-grained, host-driven offload pays off only when the accelerator configuration stays fixed: reconfiguring the device between tasks costs more than the processing itself, so that host CPUs outperform the DPU on small or variable-sized inputs. Our evaluation shows that per-invocation overhead also limits host-driven offload on computational storage: when the host invokes a function for each chunk of stored data, the cost of each invocation and of staging the chunk in device memory accounts for a large share of query execution. Rather than refining host-side offloading decisions, we argue that the iteration over data must move onto the device.
Code Representation
Both standards distinguish between functions pre-installed on the device and functions downloaded by the host at run-time. In SNIA terms, functions execute in an engine environment (CSEE) such as an operating system, a container, an FPGA bitstream or an eBPF environment. NVMe distinguishes device-defined programs from downloadable programs, and proposes eBPF for the latter, on the grounds that it is a vendor-neutral instruction set architecture [3]. In the introduction, we argued that eBPF is best understood as a relocatable binary format, used to call device-defined functions rather than to express data processing. In SNIA terms, an eBPF program that calls registered functions is an aggregator CSF, running in an eBPF CSEE and invoking CSFs that may run in other CSEEs, e.g., in hardware. In this section, we review the characteristics of eBPF that support this view and that matter for computational storage.
eBPF. BPF stands for BSD (or Berkeley) Packet Filter. It was originally proposed in 1993 to perform user-defined packet filtering, e.g., in the context of a firewall within the Unix kernel, without copying packets [33]. BPF defined by a bytecode structure together with a virtual machine. In 2014, Alexei Starovoitov introduced extended BPF (eBPF), an adaptation of BPF for Linux and modern processors [47].4 eBPF is now used to execute user-defined functions inside the Linux kernel for performance monitoring [14], security monitoring [11] and storage management [56], and as an intermediate representation for hardware-accelerated packet processing [5]. The eBPF foundation was established in August 20215 to organize the governance of this ecosystem, and the instruction set has since been standardized6.
eBPF programs. An eBPF program is composed of a single main routine, located in memory in a program slot that can be read from the eBPF run-time environment. It is a sequence of 64-bit encoded instructions, each composed of an 8-bit opcode, 4-bit destination and source registers, a 16-bit offset and a 32-bit immediate value. Opcodes are grouped into classes (load, store, ALU, byteswap and branch). ALU, byteswap and load/store operations manipulate data at 64/32/16/8 bits granularity, and a wide 128-bit encoding appends a 64-bit immediate value to a basic instruction. There are eleven registers: r0 holds the return value of a program, r1–r5 hold the arguments of called functions, r6–r9 are callee-saved, and r10 is a read-only frame pointer. The call instruction uses its immediate value as an index in a function pointer table maintained by the eBPF run-time environment. The program counter is implicit: it cannot be manipulated by eBPF instructions and is managed by the run-time environment. As a consequence, eBPF does not support indirect jumps, indirect function calls or jumps across eBPF programs7. eBPF instructions do not support floating point operations.
Compilation and linking. Both gcc and clang/LLVM have an eBPF backend that generates eBPF programs from C programs8. Figure 1 shows a simple C function that performs data massaging around a call to an external function. It takes a pointer as an argument and casts it as a struct pointer (line 6, the struct is defined in lines 1–4). It calls an external function (regfunc) on the string component of the input struct (line 7), and it writes, just beyond the input struct in memory, the product of the result of the external function with the integer element of the input struct (lines 8 and 9).

1 struct param {
2 int n;
3 char s[100];
4 };
5 int simple(void* ctx) {
6 struct param *p = (struct param *) ctx;
7 int v = regfunc(p->s);
8 ctx += sizeof(struct param);
9 * ((int*)ctx) = v*(p->n);
10 return 0;
11 }Figure 1. A C program calling an external function (simple.c).
Figure 2 is the corresponding eBPF program in LLVM assembly9. Register r1 contains the input pointer, which is first saved to the callee-saved register r6 (line 2), then incremented by the size of the integer n (4 bytes) so that it points to the s component of the input struct (line 3). r1 is the argument of the external function call (line 4). The integer value to which r6 points is then loaded into r1 (line 5), and multiplied by the return value of the external function, stored in r0 (line 6). r1 is stored at the first memory address beyond the input struct, i.e., at r6 + 104 (line 7). Finally, the return value 0 is assigned to r0 (line 8) and the program exits (line 9).

1 simple:
2 r6 = r1
3 r1 += 4
4 call regfunc
5 r1 = *(u32 *)(r6 + 0)
6 r1 *= r0
7 *(u32 *)(r6 + 104) = r1
8 r0 = 0
9 exitFigure 2. The eBPF program generated from simple.c in LLVM assembly (clang -O1 -S -target bpf simple.c).
Note that regfunc is not resolved at compile time: the bytecode refers to it by name. Note also that the compiler does not take into account the characteristics of the eBPF run-time environment, which might only allow programs of a given size or with predefined properties; for instance, the compiler can generate eBPF programs that contain loops. We can distinguish the following phases in the life cycle of an eBPF program:
Produce bytecode by compiling a C program;
Link bytecode by replacing references to external functions with their index in the function pointer map of the run-time environment, or offsets into structs with their actual offset in memory;
Load bytecode into the run-time environment;
Verify bytecode, possibly rewriting it to be safe;
Execute bytecode with a virtual machine, or as native code if it has been JIT-compiled in phase 3 or 4.
The linking phase is what makes eBPF a relocatable binary format. The code of external functions is linked with the process that runs the run-time environment. When an external function is registered, its name and function pointer are added to the external function pointer map, and a lookup by name returns its index in this map. The same bytecode can thus call a function implemented in software on one device and hardware-accelerated on another, as long as both devices register it under the same name.
Run-time environments. There exist several software implementations of the eBPF ISA. The most prominent are the eBPF subsystem within the Linux kernel, which consists of an interpreter and several JIT compilers along with a run-time environment, and uBPF, a library that implements a virtual machine with a JIT compiler, designed to run in user-space and to be embedded in kernel-space10. Hardware implementations of the eBPF ISA have also been proposed, most prominently hBPF11 and Sephirot, the eBPF processor of hXDP [5], a design specialized for packet processing on FPGA NICs. A tool is available to evaluate the conformance of an eBPF run-time environment to the instruction set12.
Memory. The stack and the program slot are regions of memory associated with the eBPF run-time environment. The size of the stack and the maximum number of instructions in the program slot are parameters of the environment. In uBPF, they are set by default to 512 B and 64K instructions respectively. In the Linux kernel, eBPF programs can exchange data with user-space, or maintain state across executions, through key-value data structures denoted maps. More importantly for us, the caller of an eBPF program usually passes a memory buffer as a context. By convention, the first argument of an eBPF program is a pointer to this context (as in Figure 1). Most eBPF programs in the Linux kernel (e.g., socket filters, tc filters, XDP programs) and all uBPF programs respect this convention. A socket filter, for instance, takes a pointer to an sk_buff, whose data_end field points to the end of the context so that its length can be computed. The context is the natural vehicle for arguments and results when eBPF programs are offloaded to computational storage: it maps onto function data memory in SNIA and subsystem local memory in NVMe.
Verification. The eBPF verifier in Linux enforces that eBPF programs terminate without using too many resources and without leaking the contents of kernel memory. It is now over 15K lines of code13. In contrast, the uBPF loader merely checks that an eBPF program fits inside the allocated program slot. The eBPF for Windows project uses the PREVAIL verifier [12] together with uBPF. Verification constraints designed for the kernel, such as bounded loops, are not necessarily appropriate for computational storage.
eBPF as a data processing language. Huang et al. [24] compare eBPF and WebAssembly as code representations for computational storage. They argue that the negative aspects of eBPF (no support for floating point operations, no support for 64-bit division) outweigh its positive aspects (low overhead, both in terms of bytecode footprint and execution time). Their comparison rests on the hypothesis that the offloaded bytecode performs the data processing itself, and it does not distinguish between device-defined functions and downloadable programs. We take a different position: eBPF programs are glue that massages the inputs and outputs of registered functions and orchestrates their execution, while the bulk of the processing is performed by native or hardware-accelerated code. Whatever its representation, however, a downloaded program remains a host-invoked function operating on data moved by the host. The code representation does not change the execution model.
Delilah
Delilah is a computational storage system that implements the execution model of the standard computational storage interfaces on an actual open-hardware device. It uses eBPF to represent the functions offloaded by the host. A first version of Delilah was presented in [19]. Here, we detail its design and implementation; Appendix A details its software and hardware components and its memory layout.
Design
Platform. We use OpenSSD Daisy [51] as hardware platform for our computational storage device. Daisy is the latest generation of OpenSSD prototypes, designed by Prof. Song and his team at Hanyang University, and commercially available through CRZ Technology. It is an accelerator card equipped with 2x100GE and PCIe Gen3x16 connectors, a Zynq Ultrascale+ Multiprocessor System on a Chip (MPSoC) and a backplane interface for connecting two M.2 SSDs. The Zynq Ultrascale+ MP-SoC combines an FPGA, denoted PL (programmable logic), with four ARM Cortex-A53 cores running embedded Linux, denoted PS (processing system). The PS only has access to peripherals (PCIe, RAM DIMMs, M.2 SSDs) through the PL. The boot image of Daisy therefore contains a PL image, i.e., an FPGA bitstream synthesized with the Vivado toolchain from a block design that combines generic IP (e.g., a DMA engine) with accelerators defined in a hardware description language, and a PS image, i.e., embedded Linux with programs cross-compiled for the ARM cores.
We use Daisy in an on-path PCIe architecture: Daisy is a PCIe device for the host and a PCIe root for the SSDs. A computational storage device accesses SSDs through their external interface. In an on-path architecture, the host thus reaches stored data through at least two DMA copies, one between host and device and one between device and SSD; in an off-path architecture, processing on the device requires the same number copies, performed with peer-to-peer DMA. In both cases, all copies go through device memory.

Figure 3. Delilah overview.
Components. The Delilah software consists of two components: the Delilah driver, a kernel module on the host, and the Delilah controller, a user-space program on the device CPU (PS). The driver exposes a character device, through which applications submit commands with io_uring; it transfers data by DMA between host memory and device memory, and passes commands to the controller through a memory region mapped in the host address space (BAR0). The controller runs a number of eBPF virtual machines, each executing an eBPF program on behalf of the host.
eBPF is the means by which the host calls device functions. A computational storage device comes with device-specific functions, implemented in software or in hardware. One option is to call each of them with a dedicated NVMe command (e.g., [9]); NVMe is extensible, but every new function then requires extending the NVMe controller. Delilah instead uses eBPF as a relocatable binary format. Device-specific functions are linked and registered with the eBPF run-time environment on the device; we denote them registered functions. To call them, the host generates an eBPF program that references them by name, and these references are resolved when the program is linked on the device. The host can thus call any registered function without extending the host-device interface.
Several eBPF run-time environments could host these programs: the Linux kernel run-time environment, uBPF in user-space, and hardware eBPF processors. The Linux kernel run-time environment is the most robust and mature, and might suit a smart NIC where data flows through the network and storage stacks inside the kernel; hardware eBPF processing might suit FPGA accelerator cards, especially in off-path architectures. The Delilah controller runs uBPF virtual machines, which give us the most flexibility to explore the design space: registered functions are ordinary user-space functions on the PS, which access the SSDs through the NVMe driver of the embedded Linux, and can in turn call hardware accelerators on the PL. Put differently, eBPF programs massage the inputs and outputs of registered functions and orchestrate their execution, as in Figure 1, while the bulk of the processing is performed by registered functions, possibly hardware accelerated, e.g., to convert Apache Parquet to Arrow in FPGA [41].
Arguments and results are exchanged through the abstraction that the eBPF run-time environment already provides: the memory buffer passed to a program as its context (§ 2.2), which we denote the data slot. Delilah exposes data slots, as well as the program slots that hold eBPF programs, as memory shared between host and device. The host writes a program into a program slot and its arguments into a data slot, requests the execution of the program on the data slot, and reads the results back from the data slot. All program and data slots have the same size, determined on the device and reported to the host through two 4-byte fields. In theory, these slots have a maximum size of 4GB each, but in practice, their size is determined by the amount of physical memory on the device that is dedicated to data transfer.
Code offloaded from the host must not be able to exploit the eBPF run-time environment to corrupt or deny access to stored data. This requires verifying the eBPF virtual machine or JIT compiler [54, 37], as well as eBPF programs and the registered functions they call. The Linux eBPF verifier does not suit computational storage: backward jumps are essential for data massaging, which makes termination analysis much harder, while termination remains necessary so that the device is not locked up by an infinite loop. Verifying offloaded code for computational storage is thus an open problem, which starts with specifying appropriate safety and security policies. We leave it out of scope: like Eid-Hermes [40], Delilah relies on the uBPF loader, which only checks that a program fits in its program slot.
Block design and device memory. The block design on the PL connects the PS to the outside world: a DMA engine over PCIe for communication with the host, memory interfaces for accessing RAM, and PCIe bridges for communicating with the M.2 SSDs. The block design used in our experiments, built from Xilinx IP modules connected through AXI interconnects (Appendix § 9.§ 2), connects the device to the host over PCIe Gen3 x8 (8 GB/s), and an M.2 SSD over PCIe Gen3 x4 (4 GB/s); a DDR4 DIMM attached to the PL holds the program and data slots.
Each DDR4 memory channel provides a peak bandwidth of 12.8 GB/s. Consider a function executed on data read from an SSD. The data is first written to device memory, then read by the PS via load instructions; the PS writes its result back to device memory via store instructions, and the host subsequently reads the result from device memory. Can the device process SSD data at 4 GB/s? The answer is no: each byte of input generates four memory accesses (two writes and two reads), requiring 16 GB/s of memory bandwidth, which exceeds the channel’s 12.8 GB/s peak, and practical sustained bandwidth is lower still. The memory access pattern is thus a bottleneck.
This pattern limits end-to-end throughput to GB/s per DIMM; with the single DIMM used in our experiments, this is the bound for the whole device, well below the 8 GB/s of the host link. Again, it is not memory bandwidth, but the way compute is attached to memory and the way data is staged in memory that constitute the bottleneck.
Delilah stages data retrieved from storage in a shared buffer, a region of device memory, distinct from the program and data slots, that is accessible to all virtual machines.
We detail how to process queries with such a computational storage platform in the next Section. Before we get there, let us focus on the Delilah implementation.
Implementation
We implemented the block design with Vivado 2019.1. The Delilah controller and the registered functions are programmed in C and cross-compiled with gcc 12.1 for PetaLinux 2019.1, the embedded operating system running on the PS. The Delilah driver is also implemented in C and compiled with gcc 12.1. Appendix A details the software components (Appendix § 9.§ 1), the hardware components (Appendix § 9.§ 2) and the memory layout (Appendix § 9.§ 3), with pointers to the code.
Transport Protocol
Driver and controller implement a transport protocol between host and device, which is a simplified form of the memory, copy and compute requests of the SNIA and NVMe APIs. Unlike the reference flow of the API, however, data is loaded from the SSDs into device memory by registered functions rather than by host-issued storage requests.
The protocol defines four commands for eBPF programs: (i) transfer an eBPF program from the host to a program slot, (ii) write program arguments to a data slot, (iii) execute an eBPF program using a program and a data slot, and (iv) read results from a data slot. The current version of Delilah supports a single command for executing an eBPF program. The rationale for defining a single command, instead of separate commands for loading a program and executing it, is to avoid a roundtrip when executing a program once. Defining separate commands for JIT compilation followed by multiple program executions with different arguments is future work. In addition to these four commands, the protocol defines three commands for clearing the device caches, clearing the controller state and returning device information.
Programs, arguments and results are transferred by DMA between host memory and the slots. The actual address of each slot is computed by the host from an offset contained in BAR0 (see Figure 14, this offset is configured by the Delilah controller when it starts). To execute a program, the driver writes the command to BAR0 and sets a control bit. The controller spawns one thread per uBPF virtual machine, pinned to its own PS core; each thread polls the control register of its virtual machine, loads the program from the program slot named in the command, and executes it on the data slot named in the command. Any virtual machine can thus execute any program slot on any data slot. On completion, the controller writes the response to BAR0 and signals the host with an MSI-X interrupt.
Figure 14. Delilah memory layout: BAR0, program and data slots, shared buffer, and placement of the uBPF virtual machines on the PS cores.
DMA Buffers and Cache Coherence
Program and data slots are DMA buffers that the controller maps into user space. They reside in the PL DDR4, which the host and the SSD access by DMA without snooping the caches of the device processors. Coherence must thus be maintained in software, by invalidating the slots before a program executes and flushing the data slot afterwards.
This leads to a trade-off. With whole-slot maintenance, the protocol is simple, but the controller, which does not know what the host exchanges, maintains entire slots, at a cost proportional to their size that dominates the latency of small programs (§ 6.3). With range-based maintenance, the execution command carries the ranges of the data slot that the host has written and will read back: the cost becomes proportional to the data exchanged, but coherence becomes part of the host-device protocol, and the host must declare its ranges correctly, at cache-line granularity. With hardware coherence, the slots would reside in the PS DDR and all DMA transfers would go through ports that the CCI-400 interconnect keeps coherent with the processor caches: neither maintenance nor declarations would be needed, but the slots would be confined to the 2 GB of PS DDR shared with Linux, and transfers would compete for the coherent ports. Hardware coherence thus trades protocol complexity for device memory capacity and bandwidth, the scarce resources on the device (§ 3.1); we have not implemented it.
Hardware-Accelerated Filtering
Registered functions can also call accelerators in the PL. As a first case, we integrated a filter synthesized with Vivado HLS [18], which supports five predicates (equal, not equal, less or equal, greater or equal, and between) and writes a position list; a later revision of the block design adds four instances of this filter (Appendix § 9.§ 2). In the controller, five registered functions, one per predicate, are a facade for a single function that drives the accelerator. Because the accelerator accesses memory directly, bypassing the caches of the device processors, this function has to do what the uBPF virtual machine does not: it translates the pointers of the eBPF program, which are relative to the data slot or the shared buffer, into physical addresses, and it flushes the input range before starting the accelerator and invalidates the output range after completion. Integrating an accelerator thus turns the coherence problem between host and device (§ 4.2) into a coherence problem between the device processors and the PL. We evaluate the accelerator in § 6.4.
Block Design Synthesis
The original version of our block design, with a 16-lane PCIe link to the host, cannot be synthesized for the Daisy FPGA because of routing congestion; reducing the number of PCIe lanes from 16 to 8 results in successful and much faster synthesis (Appendix § 9.§ 2). This points to a trade-off: any specialized block that reduces bandwidth facilitates routing without sacrificing end-to-end performance, as long as the reduced bandwidth remains above the memory bottleneck, and a block design can be optimized for throughput (with PL DIMMs) or latency (with the PS memory only). The block design should thus be considered use case specific.
Offloaded Query Processing
We hardcode a query and focus on the eBPF programs that are downloaded to the device: this is a sanity check before we invest in integrating Delilah with the storage engine of a database system. To offload query processing, we implement relational operators (e.g., filters, conversions and aggregations) as eBPF programs, which may delegate their inner loop to registered functions on the device. Filters, for instance, invoke a registered function vectorized with ARM Neon SIMD instructions, passing it the parameters of the operator (e.g., the predicate constants) read from the data slot.
The columns of the base tables are stored on the SSD, in binary format. Because device memory is the scarce resource (§ 3.1), a column cannot in general be loaded into device memory at once. We therefore split each column into chunks that fit into a shared buffer, a region of device memory, distinct from the program and data slots, into which data is staged from the SSD. Figure 4 shows the sequence of interactions for processing one chunk:

Figure 4. Chunked execution of an offloaded operator over a column split into chunks : interactions for one chunk. Steps 5–7 depend on where the chunk is (the three alternatives are shown in the frame). Solid arrows are host-issued commands; dashed arrows are data movements inside the storage subsystem.
the host driver loads the eBPF program into a program slot;
the host writes the program arguments (e.g., the column and the chunk to process, as well as the state carried over from previous chunks) to the data slot;
the host issues the execute command;
the eBPF program reads its arguments from the data slot;
if the chunk is not yet in the shared buffer, a registered function reads it from the NVMe SSD into the shared buffer, from which the operators of the current pass read it without accessing the SSD again; since a column is in general much larger than the shared buffer, it is staged chunk by chunk once per pass; the step is skipped when the chunk is already in the shared buffer, or when the operator consumes the output that the previous operator left in the data slot;
the chunk is staged in the shared buffer;
the operator reads its input: the chunk staged in the shared buffer, or the output of the previous operator in the data slot;
the operator writes its results to the data slot;
the device signals completion to the host;
the host reads the results from the data slot.
Steps 1–3, 9 and 10 are host-issued commands, while steps 4–8 correspond to data movement inside the storage subsystem. To overlap processing on the device with transfers to the host, two chunks are kept in flight: while the device processes chunk , the host fetches the results of chunk (we refer to this pattern as zig-zag).
Listings 5–7 show sanitized versions of two of the operators used for SSB Q3.3 (§ 6.4). Both follow the steps above: the eBPF program reads its arguments from the data slot (step 4), reads its input, either the chunk staged in the shared buffer or the output of the previous operator in the data slot (step 7), and writes its results to the data slot, after its arguments (step 8). The two operators differ in where their loop runs. The filter (Figure 5) is a thin eBPF program that delegates the loop to a registered function, compiled natively and vectorized with Neon (Figure 6). Each iteration of the inner loop compares four 32-bit values with the predicate constants and sets one bit per value; the loop has a fixed trip count of 32, so that a batch of 128 values yields four 32-bit bitmask words, and the native compiler can unroll it completely. The conversion (Figure 7), in contrast, runs its loop in eBPF. The eBPF program is compiled ahead of time by clang and translated instruction by instruction by the uBPF JIT, which does not optimize the code, so the loop must be tuned at the source level: the branch on the conversion type is hoisted out of the loop, and the conversion of the four digits of the year is unrolled by hand. Both operators access their input sequentially; we discuss the cost of memory accesses from the device processors in § 6.2.

Figure 5. eBPF program for the filter operator (sanitized).

Figure 6. Registered function implementing the filter loop with Neon intrinsics (sanitized; the equality filter differs only in the comparison, lines 15–16).

Figure 7. eBPF program for the conversion of dates to years (sanitized).
This execution model has a clear strength: each operator implements a loop over a single chunk, and this loop is tight on the device. However, it inherits the host-centric execution model of the standard computational storage interfaces (§ 2.1), with three consequences:
The host manages the loop across chunks. Every chunk requires a round of host-device commands, which introduces synchronization overhead.
State that must persist across chunks (e.g., partial aggregates or the positions selected by a filter) is carried as program arguments through the data slot, which may lead to pointer chasing on the device.
A query is executed as a sequence of loops, one per operator, each invoked from the host. Processing is thus column-at-a-time, in chunks, and intermediate results are materialized between operators rather than pipelined on the device.
Could the device drive the loop? One could try to avoid host-driven iteration by fusing the operators of a query into a single eBPF program that loops over
/* registered function, compiled natively */
uint32_t filter_bw_neon(uint32_t *out,
const uint32_t *in, uint32_t n,
uint32_t lo, uint32_t hi) {
uint32x4_t vlo = vdupq_n_u32(lo);
uint32x4_t vhi = vdupq_n_u32(hi);
uint32_t *start = out;
const uint32_t *end = in + n - n % 128;
while (in != end) { /* 128 values per batch */
uint32x4_t mask = vdupq_n_u32(0);
uint32x4_t bit = vdupq_n_u32(1);
/* fixed trip count (32): fully unrollable */
for (int i = 0; i < 32; i++, in += 4) {
uint32x4_t v = vld1q_u32(in);
uint32x4_t m = vandq_u32(vcgeq_u32(v, vlo),
vcleq_u32(v, vhi));
mask = vorrq_u32(mask, vandq_u32(bit, m));
bit = vshlq_n_u32(bit, 1);
}
vst1q_u32(out, mask); /* 4 x 32-bit masks */
out += 4;
}
/* remaining n % 128 values: scalar loop
with #pragma unroll(8), omitted */
return (char *)out - (char *)start;
}
int convert_year(void *ctx, int size, void *buf,
int buf_size) {
conversion_op *op = (conversion_op *)ctx;
uint32_t *out = ctx + op->padding;
/* input: chunk staged in the shared buffer */
char *in = buf + op->cached_offset;
char *end = in + op->file.size;
/* the loop runs in eBPF: dates are 8 chars
(YYYYMMDD); the branch on the conversion type
is hoisted out of the loop (other types
omitted); the digits are unrolled */
for (; in != end; in += 8)
*out++ = (in[0] - '0') * 1000
+ (in[1] - '0') * 100
+ (in[2] - '0') * 10
+ (in[3] - '0');
op->count = out - (uint32_t *)
(ctx + op->padding);
return 0;
}
all chunks on the device. Within the standard execution model, this does not work. On the input side, the program would have to read the next chunk from the SSD itself. The standard interfaces leave this to the host: SNIA storage requests and NVMe copy commands are host-issued, so a device that follows the standards strictly returns to the host for every chunk. Only the deviation of Delilah, registered functions that read from the SSD, makes device-side iteration over the input possible.
On the output side, the results of a program reach the host only through its data slot: the shared buffer is not accessible to the host, and the execution command completes only when the program terminates, so the device cannot stream partial results to the host while it keeps running. When the output of a fused program exceeds its data slot, the program must stop, the host must drain the slot, and the host must invoke the program again, with the state of the computation carried through the data slot. Iteration thus returns to the host, driven by the size of the output rather than by the size of the input. Larger data slots only push this limit back: they are bounded by device memory, and with whole-slot maintenance the cost of coherence grows with their size (§ 6.3). The final result of SSB Q3.3 is small, since it is grouped on a few cities and years, but intermediate results that the host needs, e.g., to execute joins, and the results of less selective queries, are not.
We evaluate the consequences of this execution model on the Star Schema Benchmark in § 6.4.
Evaluation
We use Delilah to answer the questions raised by its design (§ 3.1). The first three concern the mechanisms on which the host-orchestrated execution model rests: (a) what does it cost to invoke an eBPF program, and the registered functions it calls, on the device? (b) what does it cost to keep the memory shared between host and device coherent? (c) how efficiently can the host read and write device memory? The fourth question is the one this paper addresses: (d) is computational storage based on host-invoked functions and host-managed data movement a good match for query processing, and does a single hand-coded query suffice to tell?
Experimental Framework
We focus on latency, i.e., the time it takes to transfer data to and from the device and to execute eBPF programs on it. The experiments on coherence with 1 GiB data slots and on query processing use the block design of Figure 12(b) (Appendix § 9.§ 2), in which the host is connected through PCIe Gen3 x8, and the program slots, twelve 1 GiB data slots and the shared buffer reside in the PL DDR4 behind MIG_0. The micro-benchmarks of § 6.2 and § 6.3, were measured with a version of Delilah, in which four 128 MiB data slots resided in the PS DDR. The memory latencies of Figure 9 were measured from the device processors, on the PS DDR and on the PL DDR that holds the slots.

Figure 9. Average latency (in ns, log scale) of one access from the device processors to the PS DDR and to the PL DDR that holds the data slots and the shared buffer, for sequential and random reads and writes (caches enabled, 32 M accesses).

Figure 12. Delilah detailed overview: (a) software components, (b) hardware components of the PL.
We run experiments on the Daisy platform equipped with a single SSD (Samsung 970 EVO), connected to a host via PCIe Gen3 x8. The host is equipped with an Intel Core i7-4790 CPU at 3.60 GHz with 4 cores and 32 GB DDR3, and runs Ubuntu 22.04. For the query processing experiments, the host runs Linux 6.2.6, and clang 14.0 compiles the eBPF programs; the micro-benchmarks were run with Linux 5.19.17 and liburing 2.2. A development server (Intel Core i7-12700KF with 12 cores and 32 GB DDR4, Ubuntu 16.04, Linux 4.15.0) is used to synthesize the block design and to load the FPGA bitstream and OS image onto Daisy before experiments. All query processing experiments were repeated ten times, and we report averages.
For query processing, we use the Star Schema Benchmark (SSB). The columns of the base tables are stored on the SSD attached to Daisy and processed as described in § 5. We use a scale factor of 10 and one file per column.
Cost of Invoking an Offloaded Function
To measure the cost of invoking an offloaded function, we use an eBPF program that takes a file name as argument and calls a registered function that reads this file into the data slot. The size of the file varies from 1 KB to 100 MB. Figure 8 shows the end-to-end latency of the four Delilah driver commands (§ 4.1): writing the eBPF program to a program slot, writing its arguments to a data slot, executing the program, and reading the result.

Figure 8. End-to-end latency (in ms) of the four Delilah driver commands, with whole-slot maintenance (128 MiB data slots in PS DDR).
Writing the program and its arguments is fast, and reading the result takes time proportional to its size, as expected for DMA transfers. Program execution, however, has a high floor of about 16.5 ms, regardless of the size of the file. Program execution differs from the other commands in that it involves a command sent through BAR0 and the execution of the program in uBPF.
For the program that reads a 1 KB file, the latency of execution breaks down as follows. The host driver, the execution of the eBPF program and of the registered function, and interrupt handling all contribute little (0.54 ms altogether). Most of the time is spent invalidating (8.47 ms) and flushing (7.93 ms) the cache of the device processors for the DMA buffers that hold the program and data slots, 97% of the 16.94 ms end-to-end latency. Without further information, the controller maintains coherence over the entire slots, so this time is proportional to the size of the slots (128 MiB data slots in this version of Delilah), not to the amount of data. We refer to this as whole-slot maintenance.
The program used in these micro-benchmarks is deliberately minimal: it calls the registered function that reads a file into the data slot and never touches the data itself. The programs used for query processing (§ 5, Figure 5–Figure 7) are different: they scan their input, write intermediate results (bitmasks, converted years) back to device memory and, for aggregation, gather values at the positions selected by the filters. Their cost is thus dominated by memory accesses from the device processors, and on the Zynq UltraScale+ these are cheap only when they are sequential (Figure 9). With caches enabled, a sequential access takes 2–4 ns, whereas a random access takes 575–607 ns to the PL DDR that holds the program and data slots and the shared buffer in our block design (Figure 12) (and 120– 435 ns to the PS DDR), two orders of magnitude more. This is consistent with prior measurements showing that the memory performance of Zynq UltraScale+ systems varies widely, and often unexpectedly, with the access pattern and the path to memory [32]. We return to the cost of the query operators in § 6.4.
The Coherence Trade-off
Section 5.2 shows that, with whole-slot maintenance, the cost of invoking an offloaded function is the cost of keeping program and data slots coherent, regardless of how much data the program uses. This cost grows with the size of the slots and with the distance to memory: with the 1 GiB data slots in the PL DDR4, a single maintenance pass over the data slot takes about 120 ms, i.e., about 240 ms per GiB, against 130 ms per GiB in the PS DDR, consistent with the slower path to the PL DDR (Figure 9). The options of Section 3.2.2 move this cost elsewhere (question (b)). Two simple alternatives fail. Non-cacheable slots impose alignment constraints that the ELF files of eBPF programs do not respect. Copying data between separate DMA buffers and the slots beats whole-slot maintenance for small transfers (0.21 ms instead of 16.5 ms for 1 KB), but its cost grows superlinearly beyond 10 MB (1.5 s instead of 163 ms for 100 MB). Range-based maintenance makes the cost of coherence proportional to the data exchanged: with 1 GiB data slots, executing the program that reads a 1 KB file takes 0.27 ms instead of about 240 ms, but the host must declare, at cache-line granularity, the ranges it exchanges. Hardware coherence would remove both the maintenance and the declarations, at the price of confining the slots to the small PS DDR and routing all transfers through the coherent ports.
The one coherent path that Delilah already uses is BAR0, which resides in the PS DDR and is reached by the host through S_AXI_HPC0_FPD. Besides DMA, the host can thus access device memory through loads and stores to this memory-mapped region, the mechanism used to access the controller memory buffer (CMB) of NVMe devices (question (c)). Figure 10 compares the two mechanisms for reads and writes of 32 B to 4 MB. CMB writes take at most 2 µs up to 2 KB, faster than DMA, while DMA writes are faster from 4 KB on; CMB reads are on par with DMA reads only up to 128 B and slower beyond. CMB accesses need no cache maintenance, but they reach a small region of the PS DDR rather than the slots, and they are only efficient for small updates, e.g., log records [27]. This is the trade-off in miniature: coherent memory is cheap to keep coherent but scarce, whereas the large slots in the PL DDR4 must be kept coherent by the host and the controller.

Figure 10. Latency of transfers between host and device through DMA and through the memory-mapped BAR0 region (CMB). CMB writes of 32 B and 512 B took less than the 1 µs timer resolution and are not shown.
Query Processing
We now turn to question (d) and offload SSB query Q3.3 to Delilah, at scale factor 10. Q3.3 joins the lineorder fact table with the customer, supplier and date dimensions, selects customers and suppliers from two given cities over a range of years, and aggregates revenue grouped by customer city, supplier city and year.
Methodology. We hand-code the offload of Q3.3 rather than generating it from a storage manager. On this query, we run a series of experiments [18]: full offload with Neon filters (see Section 4), full offload with the FPGA filter accelerator, city filters run concurrently and sequentially, and host-only execution as a baseline. Each experiment isolates one factor, so that the breakdown below can attribute time to staging, operators, transfers and offload mechanisms.
Data layout. Each column used by the query is stored as a separate file on the SSD attached to Daisy, with one value per lineorder row. The customer and supplier cities are denormalized into the fact table and dictionary-encoded as 32-bit integers; the dictionary is kept on the host, which translates the two city names into codes. The order date is stored as 8 characters (YYYYMMDD), and the revenue as a 32-bit integer. At scale factor 10, each column holds 60 million values, i.e., 240 MB (480 MB for the order date).
Execution. We offload all the operators that are implemented as eBPF programs (full offload): the four city predicates (two per city column), the conversion of the order date into a year, the filter on the year, and the aggregation of revenue. Filters produce bitmasks, and use Neon SIMD instructions. The four city predicates run concurrently on four uBPF virtual machines, and all eBPF programs are JIT compiled. The joins are executed on the host. Because the dimension attributes are denormalized, a join reduces to combining the bitmasks returned by the filters position by position; the host then groups the qualifying positions by customer city, supplier city and year, invokes the aggregation on the device once per group, and projects the result. The intermediate results that travel to the host are the bitmasks of the filters, and the converted years used for grouping. At scale factor 10, the four columns used by the query fit in the 1 GB shared buffer: each is staged once, before the first operator, and later passes read the staged copy.
Figure 11 breaks down the runtime of the query, about 4.58 s in total, as measured in the experimental campaign of [18] (averages over ten runs). Memory accesses dominate cost: staging the four columns from the SSD into the shared buffer accounts for 1.36 s, 30% of the runtime, while the conversion operator, which reads the order dates from the shared buffer and writes the years to the data slot, accounts for 1.50 s, 33%. The filters take 0.84 s together (18%) with Neon vectorization, and aggregation 0.40 s (9%). The host-side stages, grouping (0.18 s) and cleanup (0.08 s), and the transfers of intermediate results to the host (0.23 s, 5%) account for the rest. The costs that are specific to eBPF offload (offloading programs, offloading and synchronizing the cache) are below 0.01 s.

Figure 11. Breakdown of the runtime (in s) of SSB Q3.3 (SF 10) with all eBPF programs offloaded to Delilah, with Neon SIMD filtering and the shared buffer in PL memory, in execution order. Joins, grouping and projection are executed on the host. Data from [18], read off the published figure with a resolution of 0.01 s per stage.
Hardware-accelerated filtering. We evaluated the filter accelerator of § 4.2.1. Its gains are modest. This experiment compares the accelerator with eBPF filters that do not use Neon. On the full SSB Q3.3 query, with the city filters running concurrently, hardware filtering reduces the runtime from 5.85 s to 5.72 s; with the filters run one after the other, from 7.29 s to 6.46 s. The accelerator is fast, but it shares the path to memory with the other operators, which saturates first. Developing it confirmed this: almost all of its time was spent waiting for memory, bundling accesses into vector copies brought the main speedup, whereas loop unrolling brought none and costs FPGA real estate, since unrolled iterations execute concurrently. Each accelerator also adds LUTs, flip-flops and routing pressure to a design that is already hard to place (§ 4.2.2), and debugging is difficult: an automatically inferred interface setting that forced 64-bit aligned accesses on 32-bit data was detected by neither the HLS tools nor simulation. As with the software filters, the bottleneck is the data path to memory, not computation.
Host baseline. We compare against a baseline in which the host executes the whole query [18]. Each column is read once from the SSD through Delilah (an eBPF program calls the registered function that reads the file into device memory, and the host fetches it), and all operators run on the host CPU. This baseline takes about 5.55 s. In the same experiment, full offload takes about 5.0 s, so offloading makes the query about 10% faster; with the best filter implementation (Figure 11, 4.58 s), about 17% faster. The gain does not come from processing: the host executes all its operators in about 1.5 s, whereas filtering, conversion and aggregation take about 2.7 s on the device. It comes from data movement: the host-only execution spends about 4.0 s moving the columns from the SSD through device memory to host memory, paying for coherence on each transfer, whereas offloaded operators keep the data in device memory and only ship intermediate results.
Chunked execution. The run of Figure 11 is a best case: at scale factor 10 the columns fit in the shared buffer, so each operator runs to completion over a whole column in a single invocation. Larger data sets require running each operator over chunks, as described in § 5. The experimental campaign of [18] measures this directly, by running the full offload of Q3.3 either to completion or in chunks of 128 MB. Chunking increases the runtime from about 5.0 s to about 8.1 s, i.e., by about 60%, which makes offload slower than host-only execution (5.55 s). First, every chunk requires a round of host-device commands: cache maintenance and JIT compilation are paid per chunk rather than per operator, and the device is idle while the host decides what to do next. Second, staging is no longer a separate stage but is folded into the operators that read each chunk from the SSD: the city filters take about 1.7 s and the conversion about 3.1 s, instead of 1.5 s. Third, aggregation takes about 2.1 s instead of 0.4 s, because its input arrives in chunks and it makes several passes over the same data. In addition, the shared buffer must be allocated and managed chunk by chunk, without knowing the size of the columns in advance. Chunked execution thus multiplies the costs that the model imposes: host-device round trips, coherence maintenance and passes over device memory.
The micro-benchmarks thus carry over: with range-based maintenance, invoking an eBPF program and transferring its results cost little. They do not, however, capture the cost of the operators themselves, which, unlike the program used in the micro-benchmarks, access device memory heavily (§ 6.2, Figure 9). The problem lies in the execution model. Because the host drives the loop across chunks and the sequence of operators, each operator is a separate pass over its input: in general, chunks are staged from the SSD for each pass, intermediate results are materialized in device memory or shipped to the host, and the state carried from one chunk to the next is passed as arguments through the data slot. Nothing is pipelined across operators on the device. The time is spent moving and reorganizing data within device memory, the scarce resource, rather than processing it.
Discussion
We do not claim that computational storage never pays off, but that the standard execution model does not make it pay off for query processing, and that a single query is enough to show it. First, the roadblocks are structural. Host-driven iteration over chunks, state carried through the data slot, operators invoked one pass at a time and intermediate results materialized between them are imposed by the model, so every query inherits them. Second, Q3.3 is a favorable case. It is selective, it reads few columns, its operators are simple, and at scale factor 10 its columns fit in the shared buffer, so that each is staged only once. Yet offloading saves less than a fifth of the runtime of host execution, and only because it avoids data transfers to the host; the operators themselves run slower on the device. Neither faster software filters nor hardware acceleration changes this, because the bottleneck is the path to device memory. Third, integration would make things worse, not better. Larger scale factors require running each operator over chunks, which already makes Q3.3 about 60% slower than running each operator to completion, and slower than host-only execution (§ 6.4); wider plans would multiply host-device round trips. Other queries would change the relative weight of the operators, but not the structure of the execution, which is set by the model. We therefore stopped before integrating Delilah into a storage manager and running more complete workloads: further engineering in this direction would measure the same roadblocks at a larger scale.
Perspectives
Delilah implements computational storage as exposed by the standard interfaces, with one deviation: registered functions, rather than host-issued storage requests, load data from the SSDs into device memory. This deviation takes the host out of the movement of base data, so a device that follows the standards strictly would incur at least the costs we measured. The failure we observe stems from host-driven iteration, which Delilah shares with the standard model. Our experience with it leads to lessons at three levels: the representation of offloaded code, the execution model, and the architecture of the device itself.
eBPF is glue, not the program. In Delilah, eBPF plays the role of a relocatable object file. An eBPF program is linked on the device against registered functions. This way, the host can call any registered function without extending the host-device interface, and the cost of the eBPF program itself is small. The registered functions, however, which do the actual work, are not implemented in eBPF. They are native object files, compiled for the ARM cores or bound to accelerators on the FPGA, and linked into the controller when the device is configured. A query offloaded to Delilah is therefore expressed in two representations: downloadable eBPF glue, and device-defined functions in native code, with different toolchains, life cycles and safety properties.
Memory belongs in the protocol. What decides performance is memory. On a device whose processors access shared buffers through non-coherent caches, minimizing copies and keeping DMA buffers coherent pull in opposite directions. Declaring the ranges exchanged with the device makes the cost of coherence proportional to the data exchanged, but makes how device memory is used part of the host-device protocol; hardware coherence hides it from the protocol, but confines shared buffers to the small memory of the device processors.
The host should not drive the iteration. Our main lesson concerns the execution model: Computational storage based on host-invoked functions and host-managed data movement is not a good match for query processing. When the host manages the loop across chunks, carries state from one chunk to the next, and invokes operators one after the other, execution degenerates into column-at-a-time processing in chunks, and the time goes into staging and reorganizing data in device memory. Batch requests would reduce the number of interactions but not this structure. The device must manage the iteration over data and its movement, as anticipated by SNIA’s indirect usage and variable-sequence aggregator functions, and as demonstrated by programmable SSDs over proprietary interfaces [15].
A general-purpose software stack is hard to scale. Moving the iteration onto Delilah would not be enough, because its software stack would then sit on the critical path of every I/O. Even when the host declares the ranges it exchanges, invoking an offloaded program costs on the order of a quarter of a millisecond, spent in Linux, in the controller, in uBPF and in interrupt handling on the device. These costs are specific to our platform, with Cortex-A53 cores, a general-purpose operating system, a virtual machine in user space and interrupt-driven completion; faster cores, polling, batching or kernel bypass would reduce them, perhaps by an order of magnitude.
The program representation does not fit the device. eBPF was designed for short, stateless functions that run to completion on one event. The programs that query processing offloads are different: they are long-running, they keep state across chunks (selection vectors, hash tables, partial aggregates), and they must exploit all the cores of the device concurrently. Delilah bridges this gap outside eBPF. Each uBPF virtual machine runs in a Linux thread pinned to a core, concurrency is obtained by running independent programs on different virtual machines, and state is carried across invocations through the data slot. The representation of offloaded code thus captures neither the concurrency nor the state of the computation, which are left to the operating system and to the host. Going forward, two changes appear necessary. First, as mentioned above, downloadable and registered functions should share a unified representation, so that the same toolchain, linking model and verification apply to the glue and to the operators, and so that the boundary between device-defined and downloadable code can move as accelerators evolve. Second, this representation should map efficiently onto the cores and accelerators of the device: it should express concurrent, stateful programs, with explicit state that persists across events and explicit communication between computation sites, rather than isolated functions scheduled by a general-purpose operating system.
A single computation site misses the flow. The standards model a computational storage device as a computation site: an engine, with its execution environment, executes a function on data staged in function data memory. Delilah faithfully reproduces this store-then-process structure. Data in an SSD, however, is not a buffer but a flow [30]. It moves in packets, from flash channels through the controller and its memory to the host interface, and each stage of this path offers a natural place to compute: per-channel pattern matching, record parsing and filtering in the controller, aggregation and re-encoding before the host interface. Like packet processing on smart NICs, near-data processing on SSDs calls for multiple computation sites along the data path, each with a small budget per packet, rather than a single general-purpose engine at the end of a staging step. Staging data in a single memory to compute on it is precisely what makes memory the scarce resource.
Towards co-designed SSDs. These lessons lead us from computational storage, which accesses SSDs through their external interface and has no advantage over the host when accessing stored data [28], to SSDs co-designed with the data systems they serve [29]. In the CHORYS project, we pursue this direction with OpenSSD-V [4], an open computational SSD in which RISC-V cores coupled with data-pattern accelerators replace the fixed-function controller. Its platform places several compute sites along the path between the host interface and the flash interface, and an NVMe command can be guided through an accelerated pipeline that, e.g., parses a Parquet stream, filters the records and re-encodes the survivors, so that the I/O returns the result of a pushed-down operator rather than raw blocks. The iteration is driven by the data path rather than by the host. The control path is designed for high event rates, with hardware sequencers that configure accelerators without per-task register writes and lightweight signalling that avoids trapping into a kernel. On the host side, operators exchange columnar batches according to Substrait plans, so that a device can return filtered batches, bitmaps or partial aggregates without changing the plan.
Some lessons of Delilah carry over directly. A relocatable format remains the right way to deliver code to the device, but it should cover both downloadable and device-defined functions, and describe concurrent, stateful programs that configure the compute sites, rather than functions interpreted or dispatched per event. Memory, and its coherence, must be designed into the device and its interface rather than left to the host. And safety, which Delilah left out of scope (§ 3.1), becomes more pressing when the SSD is programmable.
Conclusion
Computational storage is now standardized. The SNIA programming model is broad, but the standard interfaces that realize it, the SNIA Computational Storage API and the NVMe Computational Programs and Subsystem Local Memory command sets, expose a narrower execution model: the host invokes functions on the device, on data that the host has moved into device memory. Both standards single out eBPF to represent the functions that the host downloads at run-time. In this paper, we asked whether this form of computational storage is a good match for query processing.
To answer this question, we designed and implemented Delilah, a computational storage system that follows the host-orchestrated execution model on an actual device, the OpenSSD Daisy platform, and runs eBPF programs that call registered functions on the device. We hand-coded the offload of a single SSB query as a first step toward integrating Delilah into a storage manager, and ran a series of experiments on it. That step was sufficient. Our evaluation leads to three conclusions. First, eBPF is best understood as a relocatable binary format: it is a lightweight mechanism for calling registered, possibly hardware-accelerated, functions without extending the host-device interface, and its overhead is negligible. Second, the mechanisms of the execution model are dominated by memory, not compute. The coherence of the buffers shared between host and device dominates the latency of offloaded functions. Keeping them coherent is a trade-off: software maintenance costs time proportional to the size of the shared buffers, unless the host declares the ranges it exchanges, which makes coherence part of the host-device protocol; hardware coherence confines the shared buffers to the small memory of the device processors. Third, host-invoked functions and host-managed data movement are not a good match for query processing. When the host drives the iteration over chunks and the sequence of operators, execution degenerates into column-at-a-time processing in chunks, and the time goes into staging and reorganizing data in device memory rather than into processing it. On SSB Q3.3, offloading saves less than a fifth of the runtime of host execution, and only because it avoids moving the data to the host: the operators themselves run slower on the device. Running the operators over chunks, as larger data sets require, makes the offloaded query about 60% slower, and slower than host execution. These roadblocks are structural. They appear on a query that is favorable to offload and with data that fits in device memory, and storage-manager integration would only amplify them. They are a show-stopper for host-orchestrated computational storage in query processing, which is why we did not pursue this direction further.
Query processing on computational storage thus requires an execution model in which the device manages the iteration over data and its movement. The SNIA programming model anticipates such a model with indirect usage and variable-sequence aggregator functions, but the standard interfaces do not expose it yet. More importantly, a single general-purpose computation site, running an operating system and an eBPF virtual machine, is hard-pressed to sustain the millions of events per second of modern flash and PCIe, and it ignores the flow nature of data on its way from flash to host. We therefore argue for SSDs co-designed with data systems, with multiple computation sites along the data path, which is the direction we now pursue with OpenSSD-V.
Delilah Implementation Details
This appendix details the implementation of Delilah: its software components on host and Zynq PS, the hardware components of the PL, and the layout of device memory. The code of all components is available in the Delilah project on GitHub14; we give a pointer to the repository of each component below. This appendix, generated with Claude Code from the code base, is essentially a guide to the Delilah github repository. The scripts and programs used for the experimental evaluation are available in a separate repository15.
Software Components
Figure 12(a) shows the software components of Delilah (white) on host and Zynq PS, the components they rely on (coloured), and the application that uses them on the host (yellow). Dark and light grey denote user space and kernel.
Host: API and driver. The application calls the Delilah API, which submits commands through io-uring to the Delilah driver16. The driver is a single kernel module that consists of two components, linked with the XDMA library: cdev, which exposes a character device, and mod, which registers the PCIe device, issues DMA transfers over XDMA channels and writes command requests to BAR0. The execution of a command on the driver proceeds as follows:
The user program sends an execute command to the Delilah driver through liburing.
cdev receives the submission queue entry, including relevant buffers and metadata, as well as the ranges of the data slot to invalidate and flush (Section 3.2.2), and passes it to mod.
mod writes the command, including these parameters, to BAR0 (Appendix A.3).
mod sets the command execute bit of BAR0.
Upon completion, the device writes the command response to BAR0 and raises an MSI-X interrupt. The interrupt callback of mod, registered with XDMA, reads the command response and triggers a completion in liburing.
Device: controller. The Delilah controller17 is a user-space program on the PS, which runs on an embedded Linux image built with the PetaLinux configuration of the Delilah project18. It first configures the PCIe device and the eBPF run-time environment, then processes the commands sent by the host. It maps BAR0, the program and data slots and the shared buffer into user space with udmabuf19 (magenta), writes the header of BAR0, and spawns a thread for each uBPF virtual machine20 (cyan), which allocates the virtual machine state, registers the registered functions, and waits for commands from the host. As the PS has 4 cores, we allocate four uBPF virtual machines, and pin the thread of virtual machine to core (Figure 14).
Each thread polls the command control register of its virtual machine. When the host requests the execution of an eBPF program, the thread loads it from the program slot named in the command and executes it on the data slot named in the command: a pointer to the data slot is passed as argument to the program, and the shared buffer is accessible to all virtual machines. The stack of each virtual machine has a fixed size, set when the controller is built. eBPF programs assume a 512-byte stack by default, which may be limiting for complex offloaded programs. We submitted a patch to LLVM, which now allows overriding it [17].
On completion of a command, the PCIe DMA/Bridge IP must receive a signal on a given pin. The controller sends this signal through an AXI GPIO block, accessed through /dev/mem (Appendix § 9.§ 2), which XDMA-2 turns into an MSI-X interrupt on the host; the controller runs one IRQ thread per virtual machine.
Device: registered functions. Registered functions run in user space on the PS. They access the SSDs through the NVMe driver of the embedded Linux, and the HLS filters (Appendix § 9.§ 2) through UIO, using the registers exposed by the HLS-generated driver. For the filters, five registered functions, one per predicate, are a facade for a single function that translates the pointers of the eBPF program, which are relative to the data slot or the shared buffer, into physical addresses; flushes the input range before starting the accelerator and invalidates the output range after completion, restricting cache maintenance to the ranges it reads and writes; and drives the accelerator, polling for completion.
Hardware Components
Block design. Figure 12(b) shows the block design used in our experiments21, built from Xilinx IP modules and connected with each other and with the Zynq PS (grey) through AXI interconnects: paths to the PL DDR4 (thick solid blue), into the Zynq PS (dashed vermilion), and from the PS master port M_AXI_HPM0_FPD (thin black). The PCIe DMA/Bridge XDMA-2 connects the device to the host over PCIe Gen3 x8 (8 GB/s); XDMA-0, configured as a PCIe root port, connects an M.2 SSD over PCIe Gen3 x4 (4 GB/s); the memory interface generator MIG-0 controls a DDR4 DIMM attached to the PL, which holds the program and data slots and the shared buffer. The block design also instantiates a second SSD bridge and a second memory controller, which our experiments do not use and which Figure 12(b) omits. Host DMA enters the PS through the coherent port S_AXI_HPC1_FPD, host accesses to BAR0 through the AXI4-Lite master M_AXI_LITE and S_AXI_HPC0_FPD, and SSD DMA through the non-coherent port S_AXI_HP1_FPD. Through the master port M_AXI_HPM0_FPD and an AXI interconnect, the device processors access the PL DDR4 through MIG, and control XDMA, the AXI GPIO and the HLS filters. The AXI GPIO drives the user interrupt lines of XDMA, which the host receives as MSI-X interrupts.
HLS filters. The HLS filters22 (dashed in Figure 12(b)) belong to a later revision of the block design, which adds four instances of the filter, clocked at 333 MHz, i.e., faster than the rest of the PL; their path to the PL DDR4 is not shown. The filter supports five predicates (equal, not equal, less or equal, greater or equal, and between). It reads its input and writes a position list through an AXI master port, and receives its parameters (addresses, number of values, predicate and constants) through an AXI4-Lite control port. It processes the input in vectors of 256 32-bit values staged in on-chip BRAM, so that it issues 1 KB memory requests instead of one request per value, and compares the values of a vector in a pipelined loop.
Synthesis. The original version of our block design, with a 16-lane PCIe link to the host, cannot be synthesized for the Daisy FPGA because of congestion issues. The variant without PL DIMMs can be synthesized, but it takes 24 hours. Figure 13 shows the floor plan of the Daisy FPGA and an analysis of the vertical congestion for two block designs. When congestion exceeds 100%, Vivado starts looking for alternate routes to complete routing. Our original block design (middle) has maximum congestion of 325% (red) over a wide area, which causes routing to fail, because no alternative route can be found that meets the timing constraints. Reducing the number of PCIe lanes from 16 to 8 reduces congestion significantly (right), which results in successful and much faster synthesis; this is the block design of Figure 12(b). With 8 lanes, the host link still delivers 8 GB/s, more than the 3.2 GB/s that a DIMM sustains when each byte is accessed four times in memory (§ 3.1). A block design for writing data in bursts (e.g., writing objects in batch [9]) might instead favor host-to-memory bandwidth over SSD-to-memory bandwidth.

Figure 13. Top view of the Daisy FPGA (left), congested block design (middle) and block design without congestion (right).
Memory Layout
Figure 14 shows BAR0, the program and data slots and the shared buffer in device memory, and the placement of the uBPF virtual machines on the PS cores.
BAR0. BAR0 is the PCIe base address register that contains the PCIe device id and defines a non-prefetchable 32 MiB aperture, memory-mapped in the host address space. It resides in the PS DDR, and the controller maps it into user space. It holds:
a header, written by the controller when it starts, with the number of uBPF run-time environments, i.e., the number of uBPF virtual machines spawned by the controller, the number, base address and size of the program slots and of the data slots, and the size of the shared buffer;
for each uBPF run-time environment:
a command (commands[]), written by the host, composed of an opcode, an id and command-specific configuration bits (for the execution command, the program slot, the data slot, the program length, and the ranges of the data slot to invalidate and flush);
a response, written by the device, containing a status code when the command is executed;
a command control register (cmdctrl[], with exec and done bits), used by the host to signal the device that a command is ready to be processed and by the device to signal that the command has been executed.
Slots and shared buffer. The program and data slots and the shared buffer reside in the PL DDR4 behind MIG. The host computes the device address of a slot from the header, e.g., the base address of the data slots plus times their size for data slot , and transfers programs, arguments and results by DMA to and from that address. The shared buffer is only accessible from the device. The worker thread of uBPF virtual machine is pinned to core , polls cmdctrl[], and executes the program slot on the data slot named in commands[]. Because the host and the SSD access the slots by DMA without snooping the caches of the device processors, the controller keeps them coherent in software (§ 4.2).
References
- [1]R. Balasubramanian, J. Chang, T. Manning, J. H. Moreno, R. Murphy, R. Nair, and S. Swanson. Near-Data Processing: Insights from a MICRO-46 Workshop. IEEE Micro, 34(4):36–42, July 2014. Conference Name: IEEE Micro.
- [2]A. A. Barbalace and J. Do. Computational Storage: Where Are We Today? In Conference on Innovative Data Systems Research (CIDR), Jan. 2021.
- [3]S. Bates. NVMe Computational Storage – A update on the standard, June 2022.
- [4]P. Bonnet, A. Lerner, D. Zoni, D. Habich, E. Kervennic, G. Anchisi, G. Grandi, J. Pietrzyk, M. Wokke, N. Omer, N. Neves, N. Roma, P. Tomás, W. Lehner, and Z. István. Near-data processing in CHORYS. Datenbank-Spektrum, 2026. To appear.DOI
- [5]M. S. Brunella, G. Belocchi, M. Bonola, S. Pontarelli, G. Siracusano, G. Bianchi, A. Cammarano, A. Palumbo, L. Petrucci, and R. Bifulco. hXDP: Efficient software packet processing on FPGA NICs. Communications of the ACM, 65(8):92–100, July 2022.
- [6]W. Cao, Y. Liu, Z. Cheng, N. Zheng, W. Li, W. Wu, L. Ouyang, P. Wang, Y. Wang, R. Kuan, Z. Liu, F. Zhu, and T. Zhang. POLARDB meets computational storage: efficiently support analytical workloads in cloud-native relational database. In Proceedings of the 18th USENIX Conference on File and Storage Technologies, FAST’20, pages 29–42, USA, Feb. 2020. USENIX Association.
- [7]S. Di Girolamo, D. De Sensi, K. Taranov, M. Malesevic, M. Bestа, T. Schneider, S. Kistler, and T. Hoefler. Building blocks for network-accelerated distributed file systems. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC ’22, pages 1–14, Dallas, Texas, Nov. 2022. IEEE Press.
- [8]J. Do, Y.-S. Kee, J. M. Patel, C. Park, K. Park, and D. J. DeWitt. Query processing on smart SSDs: opportunities and challenges. In Proceedings of the 2013 international conference on Management of data - SIGMOD ’13, page 1221, New York, New York, USA, 2013. ACM Press.
- [9]J. Do, I. L. Picoli, P. Bonnet, and D. B. Lomet. Better Database Cost/Performance via Batched I/O on Programmable SSD. VLDB Journal, 2021.
- [10]J. Do, S. Sengupta, and S. Swanson. Programmable solid-state storage in future cloud datacenters. Communications of the ACM, 62(6):54–62, May 2019.
- [11]G. Fournier, S. Afchain, and S. Baubeau. Runtime Security Monitoring with eBPF. In 17th SSTIC Symposium sur la Sécurité des Technologies de l’Information et de la Communication, 2021.
- [12]E. Gershuni, N. Amit, A. Gurfinkel, N. Narodytska, J. A. Navas, N. Rinetzky, L. Ryzhyk, and M. Sagiv. Simple and precise static analysis of untrusted Linux kernel extensions. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019, pages 1069–1084, New York, NY, USA, June 2019. Association for Computing Machinery.DOI
- [13]D. Giourokis, D. P. A. Nugroho, V. Pandey, S. Zeuch, and V. Markl. Analyzing near-network hardware acceleration with co-processing on DPUs. Proceedings VLDB Endowment, 18(13):5689–5702, Sept. 2025.
- [14]B. Gregg. BPF Performance Tools. Addison-Wesley Professional, 1 edition, Nov. 2019.
- [15]B. Gu, A. S. Yoon, D.-H. Bae, I. Jo, J. Lee, J. Yoon, J.-U. Kang, M. Kwon, C. Yoon, S. Cho, J. Jeong, and D. Chang. Biscuit: A Framework for Near-Data Processing of Big Data Workloads. In 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), pages 153–165, Seoul, South Korea, June 2016. IEEE.
- [16]Z. Guo, H. Zhang, C. Zhao, Y. Bai, M. Swift, and M. Liu. LEED: A low-power, fast persistent key-value store on SmartNIC JBOFs. In Proceedings of the ACM SIGCOMM 2023 Conference, New York, NY, USA, Sept. 2023. ACM.DOI
- [17]N. Hedam. Make BPF stack size overridable, llvm/llvm-project commit 6ee6b19. GitHub, Apr. 2023.
- [18]N. Hedam. Delilah: Efficient eBPF Offload for Integrated Data Pipelines on Computational Storage. PhD thesis, IT University of Copenhagen, 2024.
- [19]N. Hedam, M. T. Clausen, P. Bonnet, S. Lee, and K. F. Larsen. Delilah: ebpf-offload on computational storage. In Proceedings of the 19th International Workshop on Data Management on New Hardware (DaMoN ’23), pages 70–76, 2023.DOI
- [20]A. HeydariGorji, S. Rezaei, M. Torbazdehkashi, H. Bobarshad, V. Alves, and P. H. Chou. Leveraging Computational Storage for Power-Efficient Distributed Data Analytics. ACM Transactions on Embedded Computing Systems, 21(6):821–82:36, Oct. 2022.
- [21]T. Hoefler, S. Di Girolamo, K. Taranov, R. E. Grant, and R. Brightwell. sPIN: High-performance streaming Processing in the Network. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’17, pages 1–16, New York, NY, USA, Nov. 2017. Association for Computing Machinery.
- [22]T. Hoefler, A. Hendel, and D. Roweth. The Convergence of Hyperscale Data Center and High-Performance Computing Networks. Computer, 55(7):29–37, July 2022. Conference Name: Computer.DOI
- [23]J. Hu, P. A. Bernstein, J. Li, and Q. Zhang. DPDPU: Data Processing with DPUs. arXiv [cs.DC], July 2024.arxiv.org/abs/2407.13658
- [24]W. Huang and M. Paradies. An Evaluation of WebAssembly and eBPF as Offloading Mechanisms in the Context of Computational Storage. arXiv:2111.01947, 2021.arxiv.org/abs/2111.01947
- [25]I. Jo, D.-H. Bae, A. S. Yoon, J.-U. Kang, S. Cho, D. D. G. Lee, and J. Jeong. YourSQL: a high-performance database system leveraging in-storage computing. Proceedings VLDB Endowment, 9(12):924–935, Aug. 2016.DOI
- [26]M. Kwon, D. Gouk, S. Lee, and M. Jung. Hardware/Software Co-Programmable Framework for Computational SSDs to Accelerate Deep Learning Service on Large-Scale Graphs. In 20th USENIX Conference on File and Storage Technologies (FAST ’22), pages 147–164, 2022.arxiv.org/abs/2201.09189
- [27]S. Lee, A. Lerner, A. Ryser, K. Park, C. Jeon, J. Park, Y. H. Song, and P. Cudré-Mauroux. X-SSD: A Storage System with Native Support for Database Logging and Replication. In Proceedings of the 2022 International Conference on Management of Data, SIGMOD ’22, pages 988–1002, New York, NY, USA, June 2022. Association for Computing Machinery.DOI
- [28]A. Lerner and P. Bonnet. Not your Grandpa’s SSD: The Era of Co-Designed Storage Devices. In Proceedings of the 2021 International Conference on Management of Data, SIGMOD/PODS ’21, pages 2852–2858, New York, NY, USA, June 2021. Association for Computing Machinery.DOI
- [29]A. Lerner and P. Bonnet. Principles of database and SSDs design. Springer, Berlin, Germany, June 2024.
- [30]A. Lerner, P. Bonnet, K. Park, and S. Lee. Macs: One ssd, many places to compute. CIDR, 2027. To appear.
- [31]C. Lukken, G. Frascaria, and A. Trivedi. ZCSD: a Computational Storage Device over Zoned Namespaces (ZNS) SSDs, Nov. 2021. arXiv:2112.00142 [cs].arxiv.org/abs/2112.00142
- [32]K. Manev, A. Vaishnav, and D. Koch. Unexpected Diversity: Quantitative Memory Analysis for Zynq UltraScale+ Systems. In 2019 International Conference on Field-Programmable Technology (ICFPT), pages 179–187, Dec. 2019.
- [33]S. McCanne and V. Jacobson. The BSD packet filter: a new architecture for user-level packet capture. In Proceedings of the USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference Proceedings, USENIX’93, page 2, USA, Jan. 1993. USENIX Association.
- [34]J. Min, M. Liu, T. Chugh, C. Zhao, A. Wei, I. H. Doh, and A. Krishnamurthy. Gimbal: enabling multi-tenant storage disaggregation on SmartNIC JBOFs. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference, New York, NY, USA, Aug. 2021. ACM.DOI
- [35]A. Montana, Y. Xue, J. LeFevre, C. Maltzahn, J. Stuart, P. Kufeldt, and P. Alvaro. A Moveable Beast: Partitioning Data and Compute for Computational Storage, Dec. 2022. arXiv:2212.11459 [cs].arxiv.org/abs/2212.11459
- [36]O. Mutlu. Intelligent Architectures for Intelligent Computing Systems. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 318–323, Feb. 2021. ISSN: 1558-1101.arxiv.org/abs/2012.12381
- [37]L. Nelson, J. Van Geffen, E. Torlak, and X. Wang. Specification and verification in the field: applying formal methods to BPF just-in-time compilers in the Linux kernel. In Proceedings of the 14th USENIX Conference on Operating Systems Design and Implementation, OSDI’20, pages 41–61, USA, Nov. 2020. USENIX Association.
- [38]NVM Express. NVM Express® Computational Programs Command Set, Revision 1.2, 2025.
- [39]NVM Express. NVM Express® Subsystem Local Memory Command Set Specification, Revision 1.2. NVM Express, Inc., Beaverton, OR, 2025.
- [40]M. Oliveira and S. Bates. eid-hermes: An open-source eBPF accelerator for QEMU and AWS F1 instances., Sept. 2022.
- [41]J. Peltenburg, L. T. van Leeuwen, J. Hoozemans, J. Fang, Z. Al-Ars, and H. P. Hofstee. Battling the CPU Bottleneck in Apache Parquet to Arrow Conversion Using FPGA. In 2020 International Conference on Field-Programmable Technology (ICFPT), pages 281–286, Dec. 2020.DOI
- [42]E. Riedel, C. Faloutsos, G. A. Gibson, and D. Nagle. Active Disks for Large-Scale Data Processing. Computer, 34(6):68–74, June 2001.DOI
- [43]A. Ryser, A. Lerner, A. Forenchich, and P. Cudré-Mauroux. D-RDMA: Bringing zero-copy RDMA to database systems. CIDR, 2022.
- [44]Samsung Electronics. Samsung Electronics Develops Second-Generation SmartSSD Computational Storage Drive with Upgraded Processing Functionality. Samsung Semiconductor EMEA.
- [45]B. Samynathan. Computational Storage For Big Data Analytics. In Workshop on Accelerating Analytics and Data Management Systems Using Modern Processor and Storage Architectures, 2019.
- [46]Sangjin Lee, Alberto Lerner, Philippe Bonnet, Philippe Cudré-Mauroux. Database Kernels: Seamless Integration of Database Systems and Fast Storage via CXL. CIDR, 2024.
- [47]A. Starovoytov. LKML: Alexei Starovoytov: [PATCH net-next] extended BPF.
- [48]Storage Networking Industry Association. Computational Storage API Version 1.1. Technical report, Storage Networking Industry Association, May 2025.
- [49]Storage Networking Industry Association. Computational Storage Architecture and Programming Model Version 1.2. Technical report, Storage Networking Industry Association, Apr. 2026.
- [50]X. Shu, M.-X. Zhang, Y. Shan, K. Chen, J. Jiang, and Y.-W. Wu. Scalio: Scaling up DPU-based JBOD key-value store with NVMe-oF target offload. Oper Syst Des Implement, pages 449–464, 2025.
- [51]C. Technology. OpenSSD - Daisy. https://www.crz-tech.com/crz/os/daisy/. Accessed: 2026-10-14.
- [52]L. Thostrup, D. Failing, T. Ziegler, and C. Binnig. A DBMS-centric evaluation of BlueField DPUs on fast networks. In International Workshop on Accelerating Analytics and Data Management Systems (ADMS), pages 1–10, 2022.
- [53]P. Wilcox and H. Litz. Design for computational storage simulation platform. In Proceedings of the Workshop on Challenges and Opportunities of Efficient and Performant Storage Systems, CHEOPS ’21, pages 1–8, New York, NY, USA, Apr. 2021. Association for Computing Machinery.
- [54]S. Yuan, F. Besson, J.-P. Talpin, S. Hym, K. Zandberg, and E. Baccelli. End-to-End Mechanized Proof of an eBPF Virtual Machine for Micro-controllers. In Computer Aided Verification: 34th International Conference, CAV 2022, Haifa, Israel, August 7–10, 2022, Proceedings, Part II, pages 293–316, Berlin, Heidelberg, Aug. 2022. Springer-Verlag.DOI
- [55]Q. Zhang, P. A. Bernstein, B. Chandramouli, J. Hu, and Y. Zheng. DDS: DPU-optimized disaggregated storage. Proceedings VLDB Endowment, 17(11):3304–3317, July 2024.DOI
- [56]Y. Zhong, H. Li, Y. J. Wu, I. Zarkadas, J. Tao, E. Mesterhazy, M. Makris, J. Yang, A. Tai, R. Stutsman, and A. Cidon. XRP: In-Kernel Storage Functions with eBPF. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pages 375–393, 2022.