WorksComputer Science
Bayesian Sequential Optimal Experimental Design for Single-Pixel Imaging using Reinforcement Learning
Abstract
Adaptive compressed sensing (CS) aims to select the most informative measurements sequentially, which is crucial for many modern imaging systems. This task can be formulated as a Bayesian sequential optimal experimental design (sOED) problem, where each new acquisition is chosen to maximally reduce uncertainty about the unknown image given prior information and subsequent data provided by acquisitions. However, in single-pixel imaging (SPI), the large design space consists of all possible binary illumination patterns. Directly optimizing the Bayesian sOED objective over this space using conventional methods is generally challenging. In this work, we reformulate sOED for SPI as a reinforcement learning (RL) problem. In RL, an agent learns a policy through repeated experimental trials to maximize a reward, here given as the expected information gain, the most complete measure of uncertainty reduction in Bayesian OED. This avoids the need to directly optimize complex information-theoretic objectives during a measurement to obtain the next design. Instead, the result is a learned policy function that directly maps the current posterior belief about the image to the next optimal binary measurement pattern, in a computationally efficient way. Using a simple yet representative SPI imaging scenario, this work evaluates the scalability of the learned policy under two different policy parameterizations. Furthermore, we investigate how performance deteriorates when only approximate posterior information is available or the measurements are impaired by noise, and propose several strategies to introduce additional design constraints motivated by biomedical applications. The numerical experiments demonstrate that the proposed Bayesian sOED-RL framework remains robust and can be extended to more complex, large-scale SPI scenarios.
Introduction
Single-pixel imaging (SPI) emerged as an alternative to conventional array-based imaging systems, with a simplified hardware and operation in wavelengths where detector arrays are expensive or unavailable [1] as their main advantages. Unlike traditional cameras, where an entire scene simultaneously using pixel arrays is captured, here an image is sequentially acquired as follows: A spatial light pattern illuminates the scene and a single photodetector records the total light intensity being reflected back. In practice, the patterns are structured via spatial light modulators or digital micromirror devices, the latter producing binary (on/off) illumination and making binary patterns a natural fit for SPI hardware [2, 3]. A clever choice of sensing patterns and reconstruction routines enable the recovery of images from far fewer measurements than the total number of pixels [4, 5]. Known as compressed sensing (CS), the mathematical framework display which properties should signals have so as to be exactly recovered from undersampled measurements, as long as the sensing patterns satisfy certain conditions.
Early approaches chose patterns for this CS framework in a generic manner, such as random binary projections or scrambled Hadamard matrices [6], which provide reasonable results over a broad range of images. When prior knowledge about the samples that are images is available, one can improve the results by adapting the acquisitions. In an optimal experimental design (OED) framework, a representation of the possible images enables the selection of maximally informative measurement patterns, allowing an efficient image recovery [7].
While conventional OED determines a fixed set of optimal patterns before any measurements are taken, sequential OED (sOED) proposes a way more efficient strategy [8]. Instead of deploying all patterns upfront, sOED adaptively selects each new pattern during the measurement process, based on what has already been learned from previous measurements. As early measurements mainly localize where interesting features lie, later ones focus on details around those specific regions. The ’a-posteriori’ distribution, a Bayesian update of the prior knowledge by each data acquisition, results in a natural state representation that captures all available information.
However, sOED for imaging faces a severe computational challenge. On each step, one must solve an optimization problem over all possible binary measurement patterns to find the one that maximally reduces the uncertainty about the image. For an image with pixels, the possible binary patterns make conventional per-step optimization computationally prohibitive, even at modest image sizes [9, 10], limiting the practical applicability of sOED to real-world imaging problems.
To overcome this barrier, we pose sOED as a reinforcement learning (RL) problem. In this approach, rather than solving an optimization problem on each step, one looks for a parametric policy mapping the current posterior distribution directly to measurement patterns. Using the expected information gain between prior and posterior image knowledge as the reward signal, policy training consists on selecting the most informative patterns given what has already been observed along experiments. At evaluation time, the policy acts as a pattern proposal function, without needing any further optimization.
In this work, two kinds of policy parameterizations are proposed and their scalability is studied. A categorical policy assigns a probability to each of the possible binary patterns, whereas a element-wise Bernoulli policy sets each pixel’s activation probability independently. Under a controlled SPI scenario, where the optimal policy is known and information gain can be computed exactly, their performance is compared after training against random and optimal design strategies. Moreover, the replacement of the posterior distribution as a state variable by a sample-based approximate and the addition of noise to the measurements are analyzed, as well as hard and soft energy constraints over the binary projections are proposed.
The paper is organized as follows. Section 2 presents the elements of single-pixel imaging and its treatment as an inverse problem, how Bayesian optimal experimental design can be applied to it and its close relation to sequential decision making processes. Section 3 bridges this theory using a reinforcement learning abstraction, and illustrates the technical implementation details for the single-pixel imaging case. Section 4 introduces setting considerations for the proposed learning scenario and numerical results obtained in the studies, with a discussion in Section 5 and conclusions in Section 6.
Background
Single-pixel imaging as a Bayesian inverse problem
A single measurement in SPI can be described as
where is the image that one wants to recover in vector form, is a discrete binary pattern and is a noise realization, usually modeled as a normally distributed uncorrelated random variable with mean zero and standard deviation . The SPI forward model, collecting observations , is described in a matrix-vector form as
with being the forward operator, which rows consist of discrete binary patterns . Assuming that the noise from individual measurements is uncorrelated, the noise variable follows a multivariate normal distribution . When , the inverse problem of recovering from measurements is underdetermined, and usually ill-conditioned, which means that a perturbation in the image does not necessarily lead to an identifiable change in the observed data [11].
To prevent this, assumptions based on ’a-priori’ knowledge of the image can be used to regularize the possible solutions of the problem. In combination with how consistent measurements are with respect to the forward model, an updated, ’a-posteriori’ knowledge of the image is obtained. This methodology can be formalized using the Bayesian framework, where is treated as a random variable rather than merely a real-valued vector, describing the solution to Eq. (2) through the Bayes formula [12]
Here, the prior distribution encompasses the knowledge of the image before measurement takes place and can encode the properties one expects the image to have, e.g., total variation priors were proposed to model images with clearly defined edges between piecewise constant areas [13]. Note that the prior distribution is independent of the measurement operator . The likelihood distribution quantifies the probability of measuring , which contains all measurements up to step , given the image using the measurement operator . Here the noise and the unknown are mutually independent variables
The marginal likelihood , also known as model evidence, is a normalizing factor for all possible images given a particular
As a result, the posterior distribution, , combines information from the prior and likelihood distributions, assigning a probability score to each possible image given observed data . Note that due to the sequential, independent SPI measurement process, the posterior distribution is updated during acquisition as
Bayesian optimal experimental design
The aim in compressed SPI is to obtain a useful reconstruction of using measurements. An important question is which of the possible experimental designs one should choose for a given prior . Bayesian OED formalizes this problem: First, it uses the Shannon entropy as an information-theoretic measure to quantify the uncertainty about the random image before the measurement
Then, it assesses how informative a particular collection of measurements acquired with a design is by introducing the information gain as a difference between prior and posterior entropy
Using the marginal likelihood in (5) and the information gain in (8), the expected information gain provides an average uncertainty change with respect to all possible measurement realizations
BOED chooses the measurement patterns that maximize the expected information gain
where refers to an ’a-priori’ design, as this choice only relies on information available before the measurement, i.e., the prior. Now consider the case where one can design pattern sequentially during the measurement, based on the information gathered so far, i.e., ’a-posteriori’ (sequential Optimal Experimental Design, sOED). In Bayesian sOED, consider the expected information gain of using pattern for the measurement step given that the previous measurements were acquired with the design
where the marginal likelihood can be expressed as an expectation over the new measurement likelihood using the the posterior until step
Note that the prior in (8), (5) was merely exchanged for the posterior in (11), (12) and only consider the design of the following pattern . The greedy sOED , which compared to (10) depends on the unknown realization , maximizes (11) in every step
Greedy designs reduce the chain of sequential design decisions that all influence the informativeness of the final -step design into a sequence of isolated design decision steps. Although this is often computationally attractive, it is a suboptimal solution to the problem of maximizing the sum of expected information gains over all steps [14] [15]. As an alternative, the full sequential OED is the globally optimal sequence of measurement patterns that maximizes the cumulative expected information gain across all steps
Unlike greedy approaches that optimize each measurement independently, this formulation accounts for the long-term information-gathering strategy by jointly optimizing over the entire measurement sequence. The key insight is that early measurement choices influence the posterior distributions encountered in subsequent steps, creating a complex interdependence between decisions that greedy methods fail to capture.
The theoretical superiority of the full sOED strategy with respect to the greedy approach, introduced in [8], is formalized as
Inequality (15) states that the cumulative information gain achieved by the full sOED sequence is at least as large as that obtained by the greedy approach , which myopically maximizes the EIG at each individual step without considering future measurements. This gap arises because the full sOED can strategically select measurements that may have lower immediate information gain but enable more informative measurements in subsequent steps—a form of temporal credit assignment that greedy methods cannot perform. Inequality (16) further shows that the full sOED outperforms the a-priori design , which selects all patterns before observing any data and thus cannot adapt to the information revealed by early measurements. Together, these inequalities establish the full sOED as the optimal approach within the Bayesian framework, motivating the development of tractable approximation methods such as the RL-based approach presented in this paper.
Computational considerations. While the full sOED (14) is theoretically superior to the a-priori OED design (10), it comes with similar computational challenges: Taking the interdependence of the terms of the sum in (14) into account means that one also needs to optimize over the exponentially large space of binary patterns. Even for modest image sizes (e.g., ), the design space contains possible patterns at each step, making exhaustive search or traditional optimization methods computationally prohibitive.
Moreover, computing the expected information gain in (11) requires integrating over all possible measurement outcomes through the marginal likelihood in (12), which itself involves an expectation over the posterior distribution from previous measurements. In practice, these integrations are often high-dimensional and lack closed-form expressions, relying on Monte Carlo-based approximations (see Section 3.4). This computation is repeated at each of the sequential steps, making the full sOED approach (14) challenging for real-time or resource-constrained imaging applications.
These computational challenges motivate the reformulation of the sOED problem as a sequential decision-making under uncertainty, and the introduction of RL-based approaches as a principled way to tackle the sOED problem (14). Instead of solving the complex optimization problems behind sOED, one learns parametric policies that directly map posterior distributions to near-optimal measurement patterns by simulating a large number of sOED instances offline, i.e. before any real measurement takes place. The complex integrals only have to be computed during this offline training phase, not during deployment.
Sequential decision-making processes
Solving the sOED problem (14) can be seen as an instance of a Markov decision process (MDP), a mathematical model to describe sequential and time-discrete decision processes under uncertainty. In MDPs, an agent interacts dynamically with an environment in order to accumulate a reward. The state represents the current configuration of the agent and environment, and every possible decision the agent can make is represented by an action that will lead to a new state with a certain probability. The agent’s decision making is represented by a policy , which is a probability distribution in , encoding the likelihood that the agent takes the decision when in state . For every decision, the agent receives a reward . The goal in MDP is to find policies that maximize the accumulated rewards for a given time window. Very often, the agent does not have full knowledge of the state , leading to a partially observable Markov decision process (POMDP) [16].
Several methodologies describe how to find a policy that maximizes the sum of rewards from the starting time until the end of the experiment. In complex scenarios, obtaining analytical solutions for this objective or using conventional optimization schemes is often infeasible. Instead, RL methods achieve the optimization goal by updating the agent’s policy based on the feedback provided by the environment at each time. The learning process accounts not only for exploiting stepwise successful actions, but also by exploring the landscape of possible environment states [16]. Two principal classes of RL algorithms exist: value-based methods and direct policy search methods. In value-based methods, the agent learns a scalar estimate of the expected cumulative rewards from a given state onward under a fixed policy, called the value function, from repeated environment interactions and derives a policy by acting greedily with respect to this function. In direct policy search, the policy is explicitly represented by a parametric function and is updated directly via stochastic gradient ascent, i.e. policy gradient methods [17, 18].
In the sOED setting, however, the practical benefits of RL are only realized when the agent can be trained on a sufficiently large number of simulated trajectories, so that it can repeatedly experience the stochastic transitions and the associated rewards under many different policies. If the available simulation budget is too small, the agent cannot adequately explore the joint state-action space, and the resulting policy is prone to overfitting to a few sampled scenarios rather than approximating the truly optimal policy of the underlying MDP/POMDP, which substantially limits the applicability of RL-based sOED in practice.
Methods
SPI-sOED reinforcement learning framework
In this section, the sequential optimal experimental design problem for single-pixel imaging defined in Eq. (14) is formulated as a RL task and a methodology to solve it is developed. The key insight is that one must choose a complete sequence of patterns that jointly maximizes the cumulative information gain about the unknown image over the entire experiment, accounting for how each measurement shapes future decisions.
The Bayesian framework from Sec. 2.1 induces this sequential structure: At any measurement step , the current knowledge about the image is summarized by the posterior distribution , where contains all measurements up to step and contains all previously used patterns. This reduces to the prior when (see Equation (5)). When a new pattern is chosen and measurement is obtained, Bayes’ rule gives us an updated posterior , which represents the refined knowledge. This sequential updating process can be outlined using the following decision-theoretic components:
State: The posterior distribution after measurement serves as the current state . This state encodes all information from past measurements and patterns that is relevant for future decisions.
Action: The binary measurement pattern used at step becomes the action . It leads to the scalar measurement .
State transition: The new posterior distribution after measurement becomes the next state . This transition is stochastic because the measurement is corrupted by noise , but the Bayesian update rule provides the deterministic mapping from to .
Reward: The immediate reward for choosing pattern in state is the expected information gain , defined in Eq. (9). Note that this reward depends only on the current state and action , and not on the resulting new state . Computing EIG requires marginalizing over all possible outcomes , which is only tractable in small-scale settings as the one showcased on this work. For more complex settings, can be approximated by the actual information gain , which depends on the observed and thereby , cf. § 3.4.
Episode: A complete sequence of measurement steps forms an episode or trajectory , representing one complete imaging experiment from prior to final posterior.
Policy: The stochastic policy maps the current state, the posterior distribution over , to a probability distribution over binary measurement patterns.
With these definitions, the sequential optimal design problem of maximizing the cumulative expected information gain from Eq. (14) can be recast as finding an optimal policy that maximizes the total reward
where is defined as the objective function for the policy training. The expectation in (17) over the distribution of trajectories integrates the stochasticity of the sample, policy and the observation noise. A trajectory can be generated as follows: a sample is drawn from the prior , set and iterate for
The remaining question is how to represent and optimize the policy in practice. The sOED state space consists of full posterior distributions over , which are high-dimensional and continuous, making value function estimation intractable. This motivates the use of direct policy search methods, where is parameterized by a differentiable function and optimized end-to-end via policy gradient ascent on . In the following sections, the specific parameterizations of adopted for SPI-sOED and the resulting learning algorithm are detailed.
SPI design policy parameterizations
The fundamental question is how to represent this distribution over binary patterns. To construct a policy from the current posterior state, we use a feed-forward neural network that maps a state representation to a probability distribution over binary measurement patterns. We consider two policy parameterizations that share the same multilayer perceptron (MLP) backbone but differ in their output dimension and final activation function. The MLP follow the model
where denotes an affine linear mapping with parameters , a non-linear activation function and the composition of functions such that , with the layer index.
A categorical policy directly outputs a probability distribution over all possible patterns. With an output layer of units and a Softmax activation, the probability for a possible pattern given a particular past state is
where indexes all possible patterns. This approach allows the network to learn arbitrary correlations between pixel activations. However, it scales exponentially in , making it practical only for very small images.
Alternatively, each pixel activation can be modeled independently: with output units and a sigmoid activation , we obtain independent Bernoulli probabilities for the activation of each pixel , of a pattern
To sample a pattern, independent draws are sampled for each pixel from its corresponding Bernoulli distribution. This scales linearly in , making it practical for larger images. Although the network still can learn to modulate individual probabilities based on the full posterior state, it cannot model correlations between pixel activations directly. With these state representations in place, the architecture of the input layer is determined by how the state is encoded, since the model processes that representation in a compatible form as input.
State representation and approximation
The RL framework from § 3.1 requires at each step a state merging the current knowledge about the image and a reward quantifying information gained. In the SPI-sOED setting, both quantities are obtained via the Bayesian posterior
but their computation depends on whether this posterior is available in closed-form or only implicitly through sampling. As we introduced in § 2.2.1, using a stochastic variable as a state corresponds to a partially observable Markov decision process.
Explicit posterior (closed-form state). When the posterior admits a closed-form expression, the full distribution reads
as the state.
A typical example is a linear Gaussian model with prior and likelihood
for which the posterior is again Gaussian,
with fully characterising . In this regime, both the state and the expected information gain can be evaluated exactly, providing a high-fidelity learning signal.
Implicit posterior (sample-based state). For more complex priors or likelihoods, the posterior is not available in closed-form and must instead be represented implicitly via a collection of posterior samples . Among several possible ways to encode this implicit representation as a state, two natural choices are considered. The first is to pass a set of samples directly to the policy,
which preserves the complete distributional information at the cost of a fixed-size input of dimension . The second, computationally cheaper alternative is to summarize the set by its empirical mean and covariance analogues of the sufficient statistics from the closed-form Gaussian case in Eqs. (26), (25)
which compresses the posterior into a low-dimensional pair of Gaussian moments that can be processed efficiently by the policy network. We study the numerical impact of the sample size on policy learning in § 4 for the one-active-pixel prior.
Reward computation and approximation: Noiseless and noisy cases
The EIG reward is evaluated at each step of every training trajectory. Computing it exactly requires integrating over both the unknown image and the future observation , which is tractable only under specific model assumptions. Two variants appear as the posterior and the likelihood admit closed-form representations or not.
For a general prior and likelihood , the exact reward is
where the expectation is over the marginal . In general, evaluating (30) exactly is intractable because it requires integrating over both and .
Given posterior samples one can approximate the marginal as
To estimate the EIG itself, a single held-out sample is drawn , independent of the denominator particles, and use the nested Monte Carlo (NMC) estimator [19]
The one-dimensional integral in (32) is over the observation space and can be evaluated analytically or by quadrature depending on the form of the likelihood.
In in absence of noise, i.e. , the likelihood collapses to a point mass . In this case the integral over in (32) vanishes and the estimator reduces to a purely discrete sum over particles. Concretely, the forward model is evaluated at the held-out sample to obtain and at each denominator particle to form the empirical marginal likelihood. Because takes values in a finite set , the EIG estimate becomes
where is the indicator function. This is simply the negative log of the fraction of particles that predict the same observation as the held-out sample.
When the observation is corrupted by additive uncorrelated Gaussian noise, with , the likelihood becomes . One can approximate it via numerical quadrature on a uniform grid spanning with spacing . This quadrature scheme can be viewed as a deterministic variant of the nested Monte Carlo (NMC) estimator of [20], where the outer expectation over observations is resolved on a fixed grid rather than by sampling; this removes the variance of the outer loop at the cost of scaling linearly with the grid resolution , which remains tractable when the observation space is one-dimensional.
At each grid point , define the weighted likelihood as
the marginal , and the posterior weights:
where is the scalar product.
The expected posterior entropy is obtained by averaging the entropy of the posterior distribution over the observation space via quadrature:
where acts as the quadrature weight, combining the marginal density with the grid spacing. The expected information gain is then
Gaussian Prior case: D-Optimal Sequential Criterion
When the prior is Gaussian, , the forward operator is linear and the noise is Gaussian, the posterior remains Gaussian at every step with mean and covariance updated via the recursions in (25) and (26). In this setting, the model evidence for the -th observation is itself Gaussian, , with predictive variance . Since the posterior covariance is independent of the observed value , averaging over the model evidence via Gaussian moment identities yields a closed-form reward
where the second equality follows from the matrix determinant lemma [21]. This is precisely the D-optimal design criterion evaluated sequentially [22, 23]; the optimal pattern at step maximises the predictive variance of the current Gaussian belief, automatically adapting to all past measurements through the running covariance . The total EIG over steps then decomposes additively as
recovering the standard Bayesian D-optimality objective [22]. Unlike the general estimator in (37), (39) requires no outer quadrature over and no posterior samples, only the covariance update, making it exact and computationally negligible.
Pattern design including budget constraints
The framework developed so far optimises binary measurement patterns to maximise information gain, without placing any other explicit restrictions on them. In many real-world applications, however, sensing is not free: every observation consumes a physical resource, as time, energy, or radiation dose, whose accumulation can harm the very system being studied. In live-cell fluorescence microscopy, for instance, cumulative light exposure directly damages the cells through multiple photodamage mechanisms [24, 25]: excitation light generates reactive oxygen species that degrade cellular components, induces phototoxic stress responses altering gene expression, and accelerates photobleaching of fluorophores. The severity of these effects scales with total photon dose, which is simply the norm of . The dose-constrained sequential optimal experimental design for single-pixel imaging is defined as
The budget encourages the policy to focus sensing capacity on the most informative regions, naturally promoting sparse, adaptive measurement strategies.
One approach to enforce the hard constraint is to directly optimize over policies that fulfill it, e.g., by using the conditional Bernoulli policies [26], which generates each new pattern conditioned on the dose used so far. While using these policies guarantees strict dose compliance, it introduces sequential dependencies across pattern proposals, which can slow training and requires the policy to anticipate future sensing opportunities.
An alternative is to fold the cost into the objective as a soft constraint, penalizing dose expenditure directly within the policy gradient
where balances information gain against dosage used. This Lagrangian relaxation allows more flexible pattern proposals and avoids the sequential conditioning required by the hard-constrained approach, but does not guarantee strict dose compliance: A too small may lead to routinely exceeded; too large and informative patterns may be suppressed.
Learning experimental designs using parametric policies
For policy gradient methods, improving the experimental design reduces to iteratively updating the policy parameters introduced in § 3.2 to maximize the expected information gain objective by gradient ascent
where is learning rate of the parameterization update.
As computing the true gradient is intractable, different methods rely on different stochastic approximation approaches. When a differentiable policy is available, one can make use of the gradient policy theorem [27], to reformulate the gradient of (17) as:
where is the expected cumulative reward. At each step , the update is proportional to , the sum of all rewards collected from that step until the end of the experiment. This means the policy is geared towards choices that proved beneficial across the remainder of the episode. Actions that consistently produce rewards are reinforced, while those that led to diminishing returns are gradually discarded. By doing this, the expected cumulative return attributes long-term success back to the decisions that were crucial in the past.
The policy gradient update is the core step in the REINFORCE algorithm [16], which approximates the expected value in (42) by Monte Carlo estimation over trajectory samples: For each trajectory sample, the expected cumulative reward is computed for each step from that point until and using (42) the gradient in (41) is approximated as
The full REINFORCE algorithm formulated for the sOED problem is described in Algorithm 1.
Algorithm 1 REINFORCE for SPI Sequential Optimal Experimental Design
Initialize policy parameters randomly
for episode to do
sample ground truth image for this episode
// Collect trajectories
for trajectory to do
(prior distribution evaluated at )
for step to do
as described in Sec. 3.4
as described in Sec. 3.3
as described in Sec. 3.1
end for 15.
// Compute expected cumulative reward
for step to do
(sum of rewards from step )
end for
end for 21.
//Update policy using (43)
Update parameters as in (41)
end for
return Trained policy
Numerical Studies
The proposed reinforcement learning approach for sOED is validated by learning optimal measurement strategies in a simple, yet representative, numerical SPI scenario. Concepts introduced in the previous section and their impact on learning performance, such as the policy parametrization (cf. Sec. 3.6), state approximation (cf. Sec 3.3), reward approximation (cf. Sec 3.4), dose constraints (cf. Sec. 3.5) performance are illustrated. The ability of the framework to find optimal strategies under increasing measurement noise is shown.
One-active-pixel SPI scenario
Consider a structured single-pixel imaging setup in which exactly one pixel of the unknown image is active with intensity 1, as described in the left side of Figure 1. This discrete sparsity-constraint means that the possible images are reduced to the discrete set of canonical basis vectors, and one can assume that the prior accurately captures this a-priori information. In this setting, both the Bayesian posterior and the expected information gain are tractable to compute either in closed form, for the noiseless scenario, or via low-dimensional numerical quadrature in the noisy case. It provides a simple benchmark in which the reinforcement learning agent can use the exact posterior as the state and the sensing strategy can be compared against analytically optimal designs as well as against random sensing pattern. It also enables a systematic study of variability across independent policy initializations, since the discrete prior supports repeated runs with a modest computational cost.
Figure 1. The posterior is being updated using measurements of the ground truth via the proposed patterns (in green). Bisection and raster scan are considered as two possible strategies to identify the ground truth from measurements via one-active-pixel prior knowledge.
Formally, the one-active-pixel set can be described by the images , where denotes the -th unit vector in . The prior is a discrete probability measure on and can be written as
where denotes a unit point mass concentrated at . Following (3), each measurement leads to posterior that is also discrete and has the same form as the prior, just with updated weights
Thus, the posterior and the RL state can be fully represented by the -dimensional vector .
In the ideal noiseless one-active-pixel setting, a measurement of any that is supported by the prior with a binary measurement pattern yields either 0 or 1
If one measures , the active pixel then lies in the support of the illuminated mask, i.e., . Updating the prior with this likelihood information means removing all one-active-pixel images with active locations outside of the mask and renormalizing the weights to yield a uniform distribution over the remaining locations, as illustrated in the trajectories of Figure 1. Measuring accordingly removes all locations that are in the support of the illuminated mask from the posterior. Intuitively, this scenario thereby corresponds to a simple ”finding a needle in a haystack” search problem: where, out of possible locations, a particular objects sits. Every measurement one can make with a pattern corresponds to asking the question ”Is the object located in any of the illuminated positions?” and receiving a ”yes-or-no” answer.
For illustration, let us consider the case case: The prior is uniform over , leading to . The simplest case is the conventional raster scan, where one pixel at a time is illuminated in a fixed order. The sequence of patterns would be , , , . Now suppose that the true image is . The first measurement is then , which discards but leaves as possible. After the likelihood update, the posterior weights become
The second pattern yields again , ruling out and leaving only and feasible, so the posterior weights are
Only after the third measurement, with and outcome , does the posterior collapses to a single delta distribution centered on the true image, such that weights are
In this worst-case trajectory, the posterior support shrinks from four to three, then to two, and only then to a single element, illustrating that, under a uniform prior, raster scanning requires on average measurements to uniquely identify the active pixel, exhibiting a linear measurement complexity in the number of image elements .
In analogy to search problems, a bisection strategy would be more advantageous, where the patterns are chosen to half the current possible locations: In the same example with uniform prior and true image , one can consider a first pattern , which only illuminates pixels 1 and 2. The first outcome is then and after renormalizing the posterior becomes uniform over and so that the weights are
An adaptive second pattern is then , which illuminates the third but not the fourth element. The second outcome is , collapsing to a delta centered in and the weights become
shrinking from four to two and then directly to a single element in just two measurements.
Another possibility is the usage of a pattern that, on top of halving the possible locations, also keeps illuminating a partial fraction of the already measured pixels. From the previous case, a pattern represents a fixed bisection strategy, and these strategies are different from the adaptive ones in the dose budget they require and their dependence on the measurements. Fixed bisection allows via redundancy to solve for all possible images in the one-active-pixel set without redesigning their activations given the measurements obtained over the experiment.
For both cases, bisection strategies half the support of the posterior distribution at each step and, more generally, identifies the active pixel in at most measurements, reflecting its logarithmic complexity in this simplified SPI setting.
Optimal EIG strategies
For the scenario described, one can find that bisection strategies are optimal with respect to EIG. Using this result, learned policies can be compared against the strategies yielding the best achievable performance. Firstly, the EIG (9) is computed for images in the one-active-pixel set for the first measurement with . The marginal likelihood
is needed to compute the expectation over the prior and posterior entropy difference.
As illustrated in Figure 2, can only take the values 0 and 1 in the noiseless scenario: The marginal likelihood of is the ratio between the number of illuminated sites in the measuring pattern , and the number of possible illuminated sites , so that and .
Figure 2. For a uniform one-active-pixel, strategies that illuminate either one or all but one site are suboptimal in an EIG sense. This can be seen from the unbalanced posterior at the different possible outcomes . For the bisection strategy, measurements 0 or 1 have equal probability and the posterior support halves, which produces the optimal one-step EIG.
Then, the entropy for the uniform one-active-pixel prior, as described in (44), reads
Following the previous section, the posterior shrinks with respect to the uniform prior. One can compute the posterior entropy by averaging over the possible images of the set after the measurement, which leaves
With this, one can write the one-step EIG as an information gain average weighted by the marginal likelihood . Decomposing the information gain as the difference between prior and posterior entropy, one obtains
which means that is a function of . If was a continuous variable on
so the would be strictly concave w.r.t. and its unique stationary point
is the global maximum. This means that for an even , the optimal pattern illuminates half the sites of the possible image set in one step.
Because every posterior is again uniform, now over the remaining image set of size , with , the step- design problem is identical to the first with corresponding to and . The same calculation gives , and at the maximum both outcomes leave possible sites. Thereby, and the support halves at every step, leading to . Bisection is then EIG-optimal per step and identifies the active pixel in exactly measurements no matter where the active pixel is located.
Experimental settings
In all presented RL experiments, is set as the number of measurements, so that the posterior is updated twice per experiment. Furthermore, is always used in (43), i.e., the policy update is based on a single simulated trajectory. This corresponds to the setting one would use for large-scale imaging scenarios. Every RL experiment is repeated 100 times with different policy initializations to study the stability and convergence of learned designs and report the mean and 95% confidence intervals in the evaluations. Unless stated otherwise, the total expected information gain of a policy is normalized for visualization purposes, to allow for the comparison of policies that propose binary pattern strategies under different experimental setups
Policy architecture comparison
Figure 3 shows two training comparisons: the scaling of image sizes and the usage of categorical or Bernoulli pattern proposal architectures, as described in § 3.2. Categorical-based policies, described in (20), reach optimal convergence in fewer episodes than the Bernoulli-based ones, formulated in (21). Scaling the image size shows a higher requirement of episodes for convergence for the optimal EIG in two experimental steps.
Figure 3. Total normalized EIG for categorical (lighter tone, square marker) and Bernoulli (darker tone, circle marker) policies trained over different image sizes and same experimental step size . Optimal and random baselines are drawn as red and gray lines, respectively, where the maximum EIG value is calculated for that experiment, obtained using a bisection strategy in the one-active-pixel prior case, and the average EIG that a fully random policy reaches which illuminates each pixel with probability 0.5.
The normalized EIG at Figure 3 shows that Bernoulli policies are slower to train than categorical policies for all image sizes. However, Bernoulli policies can parameterize distributions over the action space with fewer trainable parameters, as the output of the network is the distribution over the pattern activations, and this output layer grows linearly with the image size . Actions sampled from the Bernoulli policies are rewarded indirectly, since there is not an output unit-to-pattern correspondence as there is in the categorical. Then, the reward carries more stochasticity than in the case of categorical policies.
In the categorical case, for an image size of the output layer has a size of units given that the design space is . Larger images result challenging for a categorical parameterization, leaving Bernoulli kind of policies for these cases despite their requirement for longer training times. The stochastic reward assignment, instead of a direct pattern-to-distribution reward, and the design space size are the trade-off variables for the framework for scaling up to large imaging problems.
Effect of sample-based approximations
We study sample-based approximations for state and reward, as introduced in § 3.3 and § 3.4. This is important for translating this framework to more complex inference problems, where the posterior is accessible only through samples rather than in closed form. The focus of this section is to assess these effects separately, while in the Supplementary Data we present a brief study of both effects in combination with respect to the posterior without approximation.
State approximation
As described in § 4.1, the posterior and RL state at time is completely described by the discrete parameter , whose entries contains the probability that the active pixel is at location . Therefore, the empirical mean state approximation computed from posterior samples, as defined in (28), converges to the true state . Here, we compare policies trained with this approximation against the exact posterior benchmark.
Figure 4 shows how approximating the state via sampling affects the learning process. Surprisingly, even the policy learned with approximating the state by a single sample reaches the optimal EIG. The inflection point around episode 6,000 marks the transition where modifying the number of samples produces a different learning performance. Strategies learned using fewer samples for the state approximation do need more episodes for reaching the optimal EIG. The CI95 plots show that the variability follows a similar behavior: Policies trained with a low number of samples need more episodes to reduce the variability, but eventually all of them to to zero.
Figure 4. Total expected information gain for policy training under sample-based state approximations, with . First row figure displays mean total EIG, second row shows 95% confidence interval width. Curve for learning with no approximation, using exact state representation is illustrated in yellow for comparison of the sampling impact. Optimal policy reward is computed exactly and displayed in red for both plots.
Reward approximation
We train policies to recover the optimal one-active-pixel strategy using the sample-based EIG estimator introduced in § 3.4 as the reward signal. This setting allows us to assess whether policies trained with the estimator in (32) can recover the strategy that is optimal under the exact EIG objective in (30). Although evaluating the exact EIG requires the marginal likelihood, which is generally intractable in more complex settings, it is available in the scenario we designed and provides a benchmark for evaluation.
In Figure 5 we observe that using the sample-based reward not only enables learning the optimal EIG strategy, it seems to even accelerate training.
Figure 5. Total expected information gain using a sample-based reward approximation, with samples and a exact state representation. Optimal rewards for each sample quantity are illustrated in red, with mean values in the first row plot and 95% confidence value widths over the second row. For comparison, EIG reward calculation using the posterior distribution representation without approximation (’No Approximation’) is displayed in yellow.
The only case displaying a slower training curve than the exact reward is the case. In contrast to Figure 4, after the inflection point in episode 6,000, using more than samples for the reward approximation show a similar trend in mean total EIG and confidence interval widths to the case where the exact reward (’No Approximation’) is used as a reward signal. Note that as in the sample-based state approximation (cf. Figure 4), the training curves converge to the exact reward curves for larger numbers of samples.
Learning under noisy measurements
In the noiseless case, the first measurement collapses the posterior to delta distributions over the regions of activity or inactivity in the case of measuring a 1 or 0. In the noisy case , this is no longer true, as noisy measurements still leave posterior probability mass over the discarded areas after the first acquisition. We now examine the impact of this on learning design policies. As a reference, we again compare their performance to the EIG of the bisection strategy, normalized as in (57).
Among the two approaches introduced in § 3.4 to treat the Gaussian likelihood modeled in (1), our proposed one-active-pixel scenario requires the usage of a numerical integration method. For the experiments displayed in Figure 6 and Figure 7, the rewards using the Bayesian quadrature are computed as described in § 3.4, with elements over the grid determined by each sample.
Figure 6. Total normalized EIG for three noisy measurement levels . First column plot displays normalized EIG and second column plot 95% confidence interval widths, averaged across 100 categorical policies realizations. Curves are normalized using random and optimal baselines for each noise level, and represented in red and green, respectively.
Figure 7. Total normalized EIG for three noisy measurement levels under constant and a linear curriculum learning for noise scheduling. Columns represent noise measurement levels and rows display the normalized means and 95% confidence interval widths. Curves are normalized using random and optimal baselines for each noise level, and are drawn in gray and green lines, respectively.
We can see that the learning behavior depends on the noise level: For a lower noise level, the policies learn slower than for the higher noise level at first but then overtake them and reach the reference normalized EIG earlier (note however, that the normalization depends on ). For higher noise levels, they reach a sub-optimal inflection point in less episodes, as we observe for , but then it takes longer to converge to the reference EIG performance.
Learning an optimal EIG strategy directly at a fixed measurement noise level can be challenging, since noisy observations yield less informative posterior updates than in the noiseless setting. We investigate curriculum learning, in which training begins with noiseless measurements and the noise level is increased linearly to the target value over the course of training. The episode-dependent noise schedule is , for , where denotes the current episode. This curriculum is compared with the constant-noise training setting of Figure 6, in which for every episode. To ensure a fair comparison, we evaluate both learned policies under the same target noise level and compute their total EIG exactly using (30).
Using , Figure 7 displays a faster learning of the linear schedule after the inflection point is reached around episode 15,000 for all
noise levels. After the first episodes, the 95% confidence intervals width shrink to zero for all , illustrating that using curriculum learning presents an performance advantage with respect to learning under constant noise. The linear curriculum learning policy is trained with cleaner measurement and evaluated in a noisier environment, which suggests that these strategies present a more stable training procedure for the proposed reinforcement learning framework.
Dose-constrained methods
In this study, we examine the two approaches to incorporate dose constraints introduced in § 3.5. In the first case, the policy is a conditional Bernoulli distribution as in [28], where the design policy proposes patterns using a dose budget along a experiment, as described in (40). Training is carried out for budgets , the number of experiment steps is and the image size is .
Figure 8 shows the performance of constrained Bernoulli policies normalized by the optimal EIG and the strategy obtained using unconstrained Bernoulli policies without training, i.e., projecting patterns at random without dose constraints, as described in § 4.3 (this results in an average dose of 4 for this setup). We observe that only the trained policy with budget cannot produce a policy that beats the unconstrained random policy, stabilizing at of the normalized EIG. policies start below the unconstrained Bernoulli random baseline but after training, these policies stabilize around a normalized EIG of with a large variability. This suggests that, even with a small budget, some of these policies produce dose-constrained strategies that are better than just drawing patterns from an unconstrained Bernoulli policy. Still, the budget is too small to reach optimal EIG strategies.
Figure 8. Total normalized EIG under dose budget constraints for conditional Bernoulli policies. First and second row display mean and 95% confidence interval width, respectively. Random baseline, in red, is computed using a unconstrained Bernoulli policy. Optimal baseline is drawn in green.
As we described in § 4.2, an adaptive policy, i.e., one that truly implements sOED, needs a budget of to reach optimal performance in this two-step experiment. We can see that the training curve indeed reaches an optimal EIG until episode 40,000 and then the performance drops in the last epochs. The policy presents the fastest learning of the optimal EIG strategy along with budget , while this dose constraint is the first reaching almost zero variability around episode 25,000. Larger budget constraints follow a similar trend, with a slower, more stable performance curve and low variability after the end of training.
Figure 9 illustrates how to incorporate the constraint as a penalty term, where is the weight between the dose budget and the EIG to be optimized. We evaluate using and plot separately the EIG normalized between the optimal EIG strategy and the unconstrained random policy, the dose budget that each policy uses at the time of projecting patterns and the normalized 95% confidence interval.
Figure 9. From top to bottom, total normalized EIG, dose budgets and normalized 95% confidence interval widths. Dose constraints are used, with random and optimal baseline strategies in gray and red, respectively. For the dose budget plot, baselines represent a fixed (gray) or an adaptive (red) policy design choice.
When using the dose as a soft constraint, different EIG performances are observed by the end of the learning process. For small in the range of , the policies converge to an optimal EIG policy that proposes designs with . Intermediate also display an optimal EIG behavior with a wider confidence interval and a mean budget . In the range of large , the dose constraint weights are so heavy that the resulting design no longer achieves optimal EIG. This constraint also proposes designs that are suboptimal for the one-active pixel scenario and the confidence intervals are the widest.
Discussion
In § 4 we illustrated the usage of the reinforcement learning framework to solve the Bayesian optimal experimental design of binary pattern for single-pixel imaging under a one-active pixel prior. For this methodology, we showed different aspects that affects the performance of the method and allow it to find optimal strategies incorporating constraints and measurement
noise.
When choosing a particular parameterization for the policy, the learning curves in § 4.4 display an inflection point that replicates over the experiments. We infer that, given the two steps we set in the proposed SPI experiment, namely projecting two binary patterns to the image of interest, learning the optimal design splits into two phases. As we examined intermediate pattern proposals along training, this behavior is likely caused by learning a bisection halving pattern in the first place and later on learning the remaining strategy. This phenomena can be related to the adaptive and fixed strategies introduced in § 4.2. At the first pattern, both strategies do not differ from each other, as they have a first bisection pattern with half of their pixels active. Moreover, the position of these active pixels is not crucial for the first pattern, but it is for the second one with respect to the first to achieve an optimal EIG step, since the input to the policy changed from being the prior in the first step to one of two possible posterior distributions in the second step. Then, the inflection point suggests a faster learning of the first pattern, where the policy training reaches a suboptimal EIG. Learning the second pattern takes more episodes for the policy due to the different possible posteriors as input for which a different optimal second pattern needs to be learned.
The sample-based state and reward studies in § 5 show the potential of RL methods in large-scale imaging scenarios, where a Bayesian approach is only feasible via sampling methods. Here, we explored the effect of using state and reward estimators to learn optimal policies in the one-active pixel scenario. Our results show that training with a surprisingly small number of posterior samples is similarly fast compared to using the exact state or reward. The downside of this approach is the additional cost to simulating trajectories, which becomes more expensive in terms of compute due to the sampling times to compute the state or the reward estimators.
Lastly, training of policies under dose constraints in § 5.4 demonstrate that better strategies than random Bernoulli patterns can be learned even for high dose constraints, i.e., low pattern activation budgets. These studies also directly demonstrate that our framework is able to learn truly adaptive policies, i.e., policies that solve the sOED problem (14), not just the a-priori OED problem (16): For both type of constraints, policies can only reach the optimal EIG with budget if they act adaptively to the first measurement, cf. § 4.2.
Conclusions
This work presents a way to formulate the design proposal of single-pixel imaging binary patterns as a Bayesian sequential optimal experimental design problem. By using a reinforcement learning framework, we proposed different ways to parameterize a design policy, and illustrated how they can be used depending on the scale of the design size and the image size. For random binary patterns, if the image size is small, a categorical policy over all possible options is preferred over a Bernoulli one due to their faster training procedure.
To investigate the scaling from proof-of-concept to large scale computational imaging scenarios, we carried out sample-based approximations using numerical simulations. Surprisingly, we found that approximating the state and/or the reward with with sample-based estimators had only minor effects on the learning dynamics, even for very low numbers of samples.
We validated the robustness of the framework by learning designs when the measurements contain different noise levels. To accelerate training in this scenario, we showed that a curriculum learning strategy can be beneficial that uses a gradual noise scaling over the training episodes, simulating a degree of difficulty from simple to complex.
Moreover, we studied two possible ways to impose constraints over the illumination dosage, enabling budget-constrained single-pixel experimental designs and proofing that the learned designs are truly adaptive.
In this work, we deliberately limited our numerical studies to a simple, small-scale scenario, for which we could deduce optimal policies in an intuitive way. This allowed us to evaluate the learned policies in a thorough way and compute many quantities that are usually not tractable. In future work, we will extend this framework to other image priors and large-scale SPI scenarios.
Data availability
The code supporting the findings of this study will be released publicly upon acceptance of the manuscript.
Ethics statement
This study did not involve research with human participants, human data, human tissue, or live animals. Therefore, ethical approval was not required.
Funding
The authors have no conflicts of interest to declare. This work was supported by the European Union (No. GA 101072354). Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the European Union nor the granting authority can be held responsible for them.
Acknowledgements
M.A.O. thanks L. Veenstra for insightful discussions on Bayesian inverse problems.
Supplementary data
Section 4.5 introduced separate studies varying the number of samples over the state and reward computation. In Figure 10 we study the effect of reusing the same samples for approximating the state and the reward. In comparison to the previous approximation studies, we observe a similar behavior to the reward approximation for , where the total EIG along training is similar with no different learning trend after the inflection point at episode 6,000. For , the learning curve is slower and the number of episodes is not enough for reaching the optimal EIG. With respect to the reward approximation alone, using seems to have a stronger impact in learning the optimal strategy, suggesting that there is a strong interplay between state and reward approximation when samples are scarce.
Figure 10. Impact of reward approximation on policy learning. Training curves showing normalized cumulative for categorical policy learning under a Monte Carlo reward approximation with posterior samples. The same samples are used to compute both state and reward approximations. Curves are averaged across 100 independent policy training realizations.
References
- [1]M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly, and R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 83–91, 2008.DOI
- [2]Z. Zhang, X. Wang, G. Zheng, and J. Zhong, “Hadamard single-pixel imaging versus fourier single-pixel imaging,” Optics Express, vol. 25, no. 16, pp. 19619–19639, 2017.DOI
- [3]R. Stojek, A. Pastuszczak, P. Wróbel, and R. Kotyński, “Single pixel imaging at high pixel resolutions,” Optics express, vol. 30, no. 13, pp. 22730–22745, 2022.DOI
- [4]D. L. Donoho, “Compressed sensing,” IEEE Transactions on Information Theory, vol. 52, no. 4, pp. 1289–1306, 2006.DOI
- [5]E. J. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Transactions on Information Theory, vol. 52, no. 2, pp. 489–509, 2006.arxiv.org/abs/math/0409186
- [6]L. Gan, T. T. Do, and T. D. Tran, “Fast compressive imaging using scrambled block hadamard ensemble,” in 2008 16th European Signal Processing Conference. IEEE, 2008, pp. 1–5.DOI
- [7]T. Rainforth, A. Foster, D. R. Ivanova, and F. Bickford Smith, “Modern bayesian experimental design,” Statistical Science, vol. 39, no. 1, pp. 100–114, 2024.arxiv.org/abs/2302.14545
- [8]W. Shen and X. Huan, “Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning,” Computer Methods in Applied Mechanics and Engineering, vol. 416, p. 116304, 2023.DOI
- [9]W. J. Welch, “Branch-and-bound search for experimental designs based on D optimality and other criteria,” Technometrics, vol. 24, no. 1, pp. 41–48, 1982.
- [10]A. Foster, D. R. Ivanova, I. Malik, and T. Rainforth, “Deep adaptive design: Amortizing sequential Bayesian experimental design,” in Proceedings of the 38th International Conference on Machine Learning. PMLR, 2021, pp. 3384–3395.arxiv.org/abs/2103.02438
- [11]O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier, and F. Lenzen, Variational Methods in Imaging, ser. Applied Mathematical Sciences. New York, NY: Springer, 2009, vol. 167.DOI
- [12]A. M. Stuart, “Inverse problems: A bayesian perspective,” Acta Numerica, vol. 19, pp. 451–559, 2010.DOI
- [13]M. Lassas and S. Siltanen, “Can one use total variation prior for edge-preserving bayesian inversion?” Inverse problems, vol. 20, no. 5, p. 1537, 2004.DOI
- [14]G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher, “An analysis of approximations for maximizing submodular set functions,” Mathematical Programming, vol. 14, no. 1, pp. 265–294, 1978.
- [15]A. Alexanderian and S. Maio, “Submodularity of the expected information gain in infinite-dimensional linear inverse problems,” arXiv preprint arXiv:2602.09285, 2026.arxiv.org/abs/2602.09285
- [16]R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press, 2018.
- [17]R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” Machine learning, vol. 8, pp. 229–256, 1992.
- [18]R. S. Sutton, D. A. McAllester, S. P. Singh, and Y. Mansour, “Policy gradient methods for reinforcement learning with function approximation,” in Advances in Neural Information Processing Systems, vol. 12, 2000.
- [19]B. T. Vincent and T. Rainforth, “The darc toolbox: automated, flexible, and efficient delayed and risky choice experiments using bayesian adaptive design,” PsyArXiv preprint, vol. 5, p. 9, 2017.
- [20]X. Huan and Y. M. Marzouk, “Simulation-based optimal Bayesian experimental design for nonlinear systems,” Journal of Computational Physics, vol. 232, no. 1, pp. 288–317, 2013.DOI
- [21]R. A. Horn and C. R. Johnson, Matrix Analysis, 2nd ed. Cambridge: Cambridge University Press, 2013.
- [22]K. Chaloner and I. Verdinelli, “Bayesian experimental design: A review,” Statistical Science, vol. 10, no. 3, pp. 273–304, 1995.
- [23]V. V. Fedorov, Theory of Optimal Experiments. New York: Academic Press, 1972.
- [24]P. P. Laissue, R. A. Alghamdi, P. Tomançak, E. G. Reynaud, and H. Shroff, “Assessing phototoxicity in live fluorescence imaging,” Nature Methods, vol. 14, no. 7, pp. 657–661, 2017.DOI
- [25]J. Icha, M. Weber, J. C. Waters, and C. Norden, “Phototoxicity in live fluorescence microscopy, and how to avoid it,” BioEssays, vol. 39, no. 8, p. 1700003, 2017.DOI
- [26]A. Attia, “Probabilistic approach to black-box binary optimization with budget constraints: Application to sensor placement,” arXiv preprint arXiv:2406.05830, 2024.arxiv.org/abs/2406.05830
- [27]R. S. Sutton and A. G. Barto, “Temporal-difference learning,” Reinforcement learning: an introduction, pp. 133–160, 1998.
- [28]A. Attia and E. Constantinescu, “Optimal experimental design for inverse problems in the presence of observation correlations,” SIAM Journal on Scientific Computing, vol. 44, no. 4, pp. A2808–A2842, 2022.arxiv.org/abs/2007.14476









