betaFeedback

WorksEngineering

Stochastic Optimisation for Scalable Ptychographic Image Reconstruction

Abstract

X-ray ptychography is a lensless coherent diffraction imaging technique capable of high spatial resolution, in which an image is reconstructed by solving a large-scale inverse problem from highly redundant far-field diffraction measurements acquired under overlapping illumination. Whilst the measurement redundancy created by overlapping illumination is essential for reconstruction fidelity, it leads to computational and memory demands that scale poorly with object size and target resolution when using conventional reconstruction algorithms. This work proposes a ptychographic reconstruction framework that reduces peak memory usage and time to convergence by leveraging stochastic optimisation techniques developed for large-scale machine learning. Specifically, the framework employs stochastic gradient descent with a decaying step size that progressively reduces the effect of gradient noise, referred to here as variance reduction (VR), with gradients that agree with automatic differentiation, to optimise the ptychographic forward model using randomly sampled subsets of diffraction measurements. For an experimental dataset of 200,000 diffraction patterns of 128×128128\times128 pixels, VR is demonstrated to require 65%65\% of the peak GPU memory and 22%22\% of the reconstruction time of the established maximum-likelihood (ML) algorithm. The VR framework incorporates the Poisson maximum-likelihood formulation and partial coherence within a unified and extensible optimisation setting. Numerical experiments demonstrate that the framework achieves reconstruction quality visually comparable to established methods, with lower peak GPU memory and a shorter time

Introduction

X-ray ptychography is a lensless imaging technique which uses coherent radiation to resolve fine-scale structure with high spatial resolution, without the use of focusing optics (Rodenburg, [38]). An image is then recovered from the resulting diffraction data by solving a large-scale inverse problem based on redundant far-field measurements acquired under overlapping illumination (Rodenburg & Faulkner, [39]).

The reconstruction problem in ptychography is inherently high-dimensional and involves the joint recovery of the object transmission function, the illuminating probe, and, in practical settings, the probe positions. A wide range of iterative algorithms have been developed to address this problem, differing in how they enforce physical constraints, model experimental noise, and trade off computational cost against reconstruction fidelity. While these methods have enabled routine high-resolution imaging, they exhibit significant limitations in terms of scalability, robustness to experimental imperfections, and memory efficiency when applied to modern large-scale datasets ([31, 7] (Yu et al. 2022)).

In particular, many established reconstruction approaches either rely on sequential local updates, which limit parallel efficiency ([12]), or require global operations over all diffraction patterns ([32]), leading to high per-iteration computational and memory costs. The computational and memory challenges associated with ptychographic reconstruction are further amplified in realistic experimental regimes, including partially coherent illumination (Yu et al., 2016), probe instability ([11]), and three-dimensional extensions such as ptycho-tomography ([17]). Due to these computational, memory, and modelling challenges, there remains a need for reconstruction algorithms that reduce computational burden while maintaining robustness and modelling flexibility.

Object and Probe Recovery

Early ptychographic reconstruction methods alternate between enforcing the overlap and modulus constraints. The Ptychographical Iterative Engine (PIE), introduced by Rodenburg (Rodenburg & Faulkner, [39]), applies these constraints sequentially to recover an object estimate under the assumption of a known probe. PIE updates local regions of the object corresponding to individual probe positions, making each update computationally inexpensive but resulting in serial object updates.

Thibault et al. introduced the Difference Map (DM) algorithm to jointly recover both probe and object ([47, 46]). DM alternates projections onto constraint sets and uses their difference to form updates, enforcing global consistency across all probe positions. Compared to the sequential local-update strategy used by PIE, DM evaluates all scan positions during each iteration. This increases the per-iteration computational cost and memory usage but improving robustness to probe uncertainty ([46]) and reducing the time to iterate through the dataset. The extended PIE (ePIE) algorithm removed the requirement for a known probe by alternating object and probe updates using local information [27]. ePIE uses one diffraction frame to calculate probe and object updates at each iteration, whereas DM uses all diffraction frames to calculate an update at each iteration. Whilst ePIE has a reduced per-iteration cost relative to DM, it remains sensitive to noise and experimental imperfections and often requires careful tuning of update parameters [30].

In contrast, statistical modelling approaches cast ptychographic reconstruction as a likelihood maximisation problem over the experimental measurements. Thibault and Guizar-Sicairos formulated a maximum-likelihood (ML) objective based on Poisson statistics of photon-counting detectors [48]. The maximum-likelihood formulation provides a principled treatment of noise but also incurs significant computational cost due to the need to evaluate gradients over all diffraction patterns. The maximum-likelihood framework, DM and ePIE algorithms were subsequently extended to a partially coherent illumination model where the probe is represented as a sum of incoherent modes [49]. Other ptychographic reconstruction schemes include Relaxed Averaged Alternating Reflections (RAAR) [26], semi-implicit Douglas–Rachford (sDR) splitting [35], least-squares ML (LSQ-ML) [33], and ADMM-based formulations [55].

Computational Scaling Challenges Ptychography requires the acquisition and processing of large numbers of diffraction patterns, and typically reconstruction costs scale unfavourably with detector size, object size, and number of probe positions [7]. The unfavourable scaling of reconstruction cost is exacerbated in three-dimensional extensions such as ptycho-tomography [6], where memory requirements and computational cost grow rapidly with sample volume and number of projections. Algorithms that process all diffraction patterns simultaneously require the entire dataset to be loaded into memory for each update, with some also requiring the storage of intermediate wavefields, gradients, and auxiliary variables. This imposes substantial memory demands and tightly couples the processing of neighbouring views, complicating parallelisation. Whilst GPU acceleration and multi-node parallelism reduce wall-clock time [12, 14] (Wakonig et al. 2020; Yu et al. 2022), these hardware-based acceleration strategies do not fundamentally improve algorithmic scalability and often require access to large GPU clusters with high memory capacity [7] (Yu et al. 2022). The memory requirements, parallelisation constraints, and dependence on large GPU resources motivate the development of reconstruction algorithms that explicitly reduce per-iteration cost and peak memory usage.

This work proposes a ptychographic reconstruction framework that reduces per-iteration cost and peak memory usage, whilst maintaining reconstruction accuracy, by combining small fixed-size mini-batches with a decaying step-size schedule [37, 3] that progressively reduces the effect of gradient noise; we refer to this as variance reduction (VR).

The contributions of this work are as follows:

  1. A mini-batch reconstruction algorithm for ptychography that minimises the Poisson negative log-likelihood with a multi-mode probe model, samples diffraction patterns in proportion to their intensity, and reduces the effect of gradient noise with a decaying step size (§ 2).

  1. Time and memory models for this algorithm and for the full-batch ML algorithm of PtyPy, with constants measured by independent benchmarks (§ 2.6).

  1. A comparison with the DM, ePIE and ML algorithms of PtyPy on three experimental datasets from Diamond Light Source, of up to 2×1052 \times10^{5} diffraction patterns (§ 4).

  1. Synthetic scaling experiments of up to 1.02×1061.02 \times10^{6} diffraction patterns that test the time and memory models (§ 5).

§ 3 describes the hardware, the reporting metrics and the convergence criterion, and § 6 summarises the findings and the questions they leave open.

Problem Definition

In a ptychography experiment, a coherent X-ray beam, referred to as the probe P\mathbf{P}, illuminates an object O\mathbf{O}, commonly modelled as a complex-valued transmission function. The interaction between the probe and object produces an exit wave-field, ψ\psi, that encodes information about the object structure.

At a given probe position, the exit wave is modelled discretely as the element-wise product of the spatially discretised probe and object,

ψi,j=Pi,j(r)⋅Oi,j(r),(1)\psi_{i,j} = P_{i,j}(\mathbf{r}) \cdot O_{i,j}(\mathbf{r}), \tag*{(1)}

where (i,j)(i,j) index spatial coordinates in the object plane. To acquire spatially localised information, the probe is scanned across the object and multiple diffraction patterns are recorded. At the kthk^{\mathrm{th}} scan position, the probe is displaced by a transverse shift rk\mathbf{r}_k, resulting in the exit wave

ψi,j(rk)=Pi,j(r−rk)⋅Oi,j(r).(2)\psi_{i,j}(\mathbf{r}_k) = P_{i,j}(\mathbf{r} - \mathbf{r}_k) \cdot O_{i,j}(\mathbf{r}). \tag*{(2)}

The scan positions are chosen to introduce overlap between adjacent illuminated regions of the object. The overlap between adjacent illuminated regions introduces redundancy in the measurements, which is central to successful ptychographic reconstruction. Each exit wave propagates from the object plane to the detector, where an intensity-only diffraction pattern II is measured. The detected intensity is modelled as

Ii,j,k=∣P[ψi,j(rk)]∣2,(3)I_{i,j,k} = \left|\mathcal{P}\left[\psi_{i,j}(\mathbf{r}_k)\right]\right|^{2}, \tag*{(3)}

where P\mathcal{P} denotes the wave propagation operator from the object plane to the detector plane. In the far-field regime, the propagation operator P\mathcal{P} is commonly approximated by a discrete Fourier transform F\mathcal{F}, up to a known complex-valued scaling factor (Kappeler et al., 2017). Since only intensities are recorded at the detector and phase information is lost, the measurement process gives rise to the phase retrieval problem. The ptychographic reconstruction task consists of recovering the probe P\mathbf{P} and object O\mathbf{O} from a collection of intensity measurements acquired at different probe positions. This can be expressed as the nonlinear inverse problem

Ii,j,k=∣F[ψi,j(rk)]∣2+ϵ,(4)I_{i,j,k} = \left|\mathcal{F}\left[\psi_{i,j}(\mathbf{r}_k)\right]\right|^2 + \epsilon, \tag*{(4)}

where ϵ\epsilon represents measurement noise, which is well modelled as Poisson-distributed for photon-counting detectors [48]. Each exit wave corresponds to a limited field of view determined by the probe support, such that the spatial dimensions of I\mathbf{I} and P\mathbf{P} are smaller than those of the reconstructed object O\mathbf{O}. Consequently, the inverse problem in Equation (4) is ill-posed and admits non-unique solutions [13]. Ambiguities arise from, for example, an arbitrary complex scaling between P\mathbf{P} and O\mathbf{O}, as well as global phase offsets. Moreover, modelling errors and experimental imperfections generally preclude an exact solution consistent with all measurements. To render the problem tractable, additional constraints derived from physical modelling and experimental design are imposed [46]. A common constraint enforces consistency with the measured diffraction intensities via the modulus constraint,

∣F[Pi,j(r−rk)⋅Oi,j(r)]∣=Ii,j,k.(5)\left|\mathcal{F}\left[\mathbf{P}_{i,j}(\mathbf{r}-\mathbf{r}_k)\cdot\mathbf{O}_{i,j}(\mathbf{r})\right]\right| = \sqrt{I_{i,j,k}}. \tag*{(5)}

A second key constraint arises from the intentional overlap between adjacent probe positions as formulated in Eq. (2), which introduces spatial redundancy into the diffraction data and is essential for resolving phase ambiguities and enabling stable reconstruction. From an optimisation perspective, ptychography can be formulated as estimating P\mathbf{P} and O\mathbf{O} that best explain the measured diffraction data under the forward model.

Given the measured intensities I\mathbf{I} and estimated probe positions rk\mathbf{r}_k, recovery of the probe and object is commonly achieved by minimizing a loss function that quantifies the discrepancy between measured and predicted intensities. The resulting optimisation problem can be written as

P,O≈argmin⁡P^,O^∑k=1KL(I^k(P^,O^),Ik).(6)\mathbf{P},\mathbf{O} \approx\underset{\widehat{\mathbf{P}},\widehat{\mathbf{O}}}{\operatorname{argmin}} \sum_{k=1}^{K} \mathcal{L}\left(\widehat{I}_k(\widehat{\mathbf{P}},\widehat{\mathbf{O}}), I_k\right). \tag*{(6)}

Here, L(⋅,⋅)\mathcal{L}(\cdot,\cdot) denotes a loss function quantifying mismatch with measured data, and P^\widehat{\mathbf{P}} and O^\widehat{\mathbf{O}} are estimates of the unknown parameters which are used to calculate an estimate of a given diffraction pattern, I^k\widehat{I}_k. Due to noise and model mismatch, the optimal solution to Eq. (6) generally corresponds to a non-zero loss when reconstructing experimental data.

Multi-Modal Reconstruction and Diffraction Pattern Scaling

The assumption of a fully coherent probe is a simplifying approximation, as X-ray sources generally exhibit partial spatial and temporal coherence [50]. Partial coherence can be modelled by reconstructing multiple incoherent probe modes [23]. In this work, we adopt the multi-modal formulation proposed by Thibault and Menzel [49]. Specifically, the probe is represented as a set of MM modes. For a given probe position, the exit waves corresponding to each mode are propagated independently and the resulting intensities are summed incoherently. The resulting modulus constraint is given by

∑m=1Mκk∣F[Pi,j,m(r−rk)⋅Oi,j(r)]∣2=Ii,j,k,(7)\sum_{m=1}^{M} \kappa_k \left|\mathcal{F}\left[P_{i,j,m}(\mathbf{r}-\mathbf{r}_k)\cdot O_{i,j}(\mathbf{r})\right]\right|^2=I_{i,j,k}, \tag*{(7)}

where κk∈R+\kappa_k \in\mathbb{R}_{+} is a per-pattern scaling factor that accounts for fluctuations in probe amplitude across the dataset. The per-pattern scaling parameters κk\kappa_k are optimized jointly with the probe and object.

Loss Function

Diffraction patterns are typically recorded using photon-counting detectors, for which measurement noise is well described by Poisson statistics [48]. While the Poisson distribution approaches a Gaussian distribution in the high-count regime, the Gaussian approximation becomes inaccurate for low-dose measurements. In the Poisson noise model, for predicted intensities I^k=I(P^,O^,rk)\hat{I}_k=I(\hat{P},\hat{O},\mathbf{r}_k), the loss is given by

L(I^k,Ii,j,k)=I^k−Ii,j,klog⁡(I^k)+log⁡(Ii,j,k!),(8)\mathcal{L}(\hat{I}_k,I_{i,j,k})=\hat{I}_k-I_{i,j,k}\log(\hat{I}_k)+\log(I_{i,j,k}!), \tag*{(8)}

where all operations are applied element-wise. Set-projection algorithms such as DM implicitly assume that the constraint sets intersect, an assumption often violated in experimental data, leading to stagnation or instability [48]. Likelihood maximisation algorithms, such as our proposed VR algorithm do not make the same assumption however. Moreover, additional physical effects can be incorporated straightforwardly by augmenting the loss function, provided the forward model remains differentiable. An advantage of likelihood-based optimisation over set-projection algorithms is that noise and model mismatch are naturally incorporated into the objective function. By default, PtyPy assumes a Gaussian noise model and minimizes the Gaussian negative log-likelihood,

L(I^k,Ii,j,k)=12σi,j,k2(I^k−Ii,j,k)2,(9)\mathcal{L}(\hat{I}_k,I_{i,j,k})=\frac{1}{2\sigma_{i,j,k}^{2}}\left(\hat{I}_k-I_{i,j,k}\right)^2, \tag*{(9)}

where the spatially varying variance is approximated as [48, 36]

σi,j,k2≈Ii,j,k+1.(10)\sigma_{i,j,k}^{2}\approx I_{i,j,k}+1. \tag*{(10)}

To ensure a fair comparison in § 4 and § 5, reconstructions in VR are performed twice for each dataset using a fixed random seed: once to record runtime and memory usage, and once to compute Gaussian NLL values for direct comparison with PtyPy.

Probe initialisation

In many experimental settings, a reasonable estimate of the probe is available, for example from prior measurements at the same instrument. In this work, we restrict attention to datasets for which an approximate probe model is provided and use this as the initial probe estimate. To initialise a multi-modal probe when only a single-mode probe estimate is available, we compute a singular value decomposition (SVD) of the single-mode probe estimate,

Pˉ0,i,j=UΣVT,(11)\bar{P}_{0,i,j}=U\Sigma V^{\mathsf{T}}, \tag*{(11)}

and retain the MM dominant components to initialize the probe modes,

Pˉ0,i,j,m=σmumvmT,m=1,…,M.(12)\bar{P}_{0,i,j,m}=\sigma_m u_m v_m^{\mathsf{T}}, \qquad m=1,\ldots,M. \tag*{(12)}

Each probe mode is normalized to unit mean magnitude,

P^0,i,j,m=i′j′Pˉ0,i,j,m∑i′=1i′∑j′=1j′∣Pˉ0,i,j,m∣,(13)\hat{P}_{0,i,j,m}=\frac{i'j'\bar{P}_{0,i,j,m}}{\sum_{i'=1}^{i'}\sum_{j'=1}^{j'}\left|\bar{P}_{0,i,j,m}\right|}, \tag*{(13)}

where i′i' and j′j' denote the probe pixel dimensions. Normalising each probe mode to unit mean magnitude improves numerical stability and enables consistent optimizer hyperparameters across datasets.

Optimisation Framework

Ptychographic reconstruction may be formulated as an optimisation problem by defining an objective function that measures the discrepancy between measured diffraction intensities and those predicted by a candidate object–probe pair as described in Eq. (6). Minimizers of this objective function are taken as solutions to the ptychographic inverse problem.

Gradient Descent Gradient descent is a first-order iterative method that updates parameters in the direction of steepest descent of the objective function. The update rule is

θl+1=θl−αl1K∑k=1K∇θL(I^(θl,k),Ik).(14)\theta_{l+1}=\theta_l-\alpha_l\frac{1}{K}\sum_{k=1}^{K}\nabla_{\theta}\mathcal{L}\left(\hat{I}(\theta_{l,k}),I_k\right). \tag*{(14)}

where θ\theta denotes the optimisation variables (the real-space pixel values of the object OO and probe PP, and the per-pattern scaling factors κk\kappa_k), ll is the iteration index, and α\alpha is the step size. The choice of α\alpha strongly influences convergence behaviour.

Automatic differentiation frameworks, such as PyTorch [34], enable efficient computation of gradients by representing the forward ptychographic model as a computational graph of differentiable operations [31]. This allows gradients of the loss function to be computed directly with respect to each object and probe pixel using only first-order information. Whilst gradient descent is straightforward to implement, it is computationally inefficient for large ptychographic datasets and often exhibits slow or suboptimal convergence [22], motivating the use of stochastic gradient descent.

Stochastic Gradient Descent. Stochastic gradient descent (SGD) reduces the cost of evaluating the gradient update over an entire dataset by approximating the gradient using a subset of the data at each iteration,

θl+1=θl−α1K′∑k=1K′∇θL(I^(θl,k),Ik),K′<K.(15)\theta_{l+1} = \theta_l - \alpha\frac{1}{K'} \sum_{k=1}^{K'} \nabla_{\theta}\mathcal{L}\left(\hat{I}(\theta_{l,k}), I_k\right), \quad K' < K. \tag*{(15)}

Here, K′K' denotes the number of diffraction patterns used to form the gradient estimate. The subset of size K′K' is termed a mini-batch, and the full dataset KK is the batch. Mini-batch gradient estimates introduce variance relative to the full-batch gradient [20], but substantially reduce per-iteration cost and have been shown to converge efficiently in large-scale inverse problems [2].

Selecting mini batches using uniform random sampling yields unbiased gradient estimates [4], but does not account for variation in information content across diffraction patterns. Patterns with higher total intensity typically contain stronger signal and greater probe information (Du et al., 2020). We therefore adopt intensity-weighted sampling, assigning selection probability

Π(k)=∥Ik∥F2∑n=1K∥In∥F2,(16)\Pi(k) = \frac{\lVert I_k \rVert_F^2}{\sum_{n=1}^{K} \lVert I_n \rVert_F^2}, \tag*{(16)}

where ∥⋅∥F\lVert\cdot\rVert_F denotes the Frobenius norm. Sampling is performed with replacement, such that the probability of selecting a given diffraction pattern is independent across mini-batches. High-intensity frames are therefore selected more often, while every frame with non-zero intensity keeps a non-zero probability of selection, so all frames can contribute to gradient updates during a reconstruction.

For the reconstructions reported here, the gradient of the Poisson negative log-likelihood with respect to the object and probe is evaluated with a hand-derived Wirtinger expression rather than by back-propagation through the computational graph. The hand-derived gradient agrees with the automatic-differentiation gradient to floating-point rounding, but avoids storing the intermediate quantities of the forward model, which lowers the peak memory of each update.

Adaptive Optimiser In this work, we employ the NAdam optimizer (Dozat, [8]), which combines adaptive per-parameter learning rates with Nesterov momentum to improve convergence speed and stability. Adaptive learning rates mitigate the imbalance in convergence rates between low- and high-spatial-frequency components observed in maximum-likelihood ptychography updates (Thibault & Guizar-Sicairos, [48]).

Variance Reduction

The method used here reduces the effect of gradient noise on the iterates by decreasing the step size. Throughout this paper, variance reduction (VR) refers to this mechanism.

In the experiments reported in § 4 and § 5 the mini-batch size K′K^{\prime} is held fixed throughout each reconstruction, resulting in reduced memory complexity compared to variance reduction with an increasing batch size. In the general variance-reduction framework however, the mini-batch size can be adjusted dynamically during optimisation. The step size used in gradient descent, α\alpha, also plays a critical role in convergence behaviour. Excessively large values amplify gradient noise and can lead to instability, while overly small values slow convergence ([43, 53]). In this work, α\alpha is decreased over successive iterations to satisfy the Robbins–Monro conditions ([37, 3]).

To make the variance-reduction mechanism explicit, the mini-batch gradient estimator glg_l may be decomposed as

gl=∇L(θl)+ξl,E[ξl∣θl]=0,(17)g_l = \nabla\mathcal{L}(\theta_l) + \xi_l,\qquad\mathbb{E}[\xi_l \mid\theta_l] = 0, \tag*{(17)}

where ∇L(θl)\nabla\mathcal{L}(\theta_l) is the full-batch gradient and ξl\xi_l represents the stochastic error introduced by unbiased mini-batch sampling. The SGD update can therefore be written as

θl+1=θl−αl∇L(θl)−αlξl.(18)\theta_{l+1} = \theta_l - \alpha_l \nabla\mathcal{L}(\theta_l) - \alpha_l \xi_l. \tag*{(18)}

Under standard bounded-variance assumptions, the variance of the stochastic perturbation satisfies

E[∥ξl∥2]=O(αl2Kl′),(19)\mathbb{E}\left[\lVert\xi_l \rVert^2\right] = \mathcal{O}\left(\frac{\alpha_l^2}{K^{\prime}_l}\right), \tag*{(19)}

where Kl′K^{\prime}_l denotes the mini-batch size at iteration ll. For our VR algorithm, K′K^{\prime} is independent of iteration number. Consequently, the stochastic component decreases as the mini-batch size increases or as the step size decreases. In the limiting case of full-batch gradient descent (Kl′=KK^{\prime}_l = K), the sampling error vanishes (ξl=0\xi_l = 0), while for fixed mini-batch size the influence of the stochastic term tends to zero as αl→0\alpha_l \to0.

Consequently, the SGD trajectory approaches that of full-batch gradient descent as either the mini-batch size increases or the step size decreases. In the present work, variance reduction is obtained primarily through a decaying step-size schedule, which progressively lowers the stochastic noise floor while retaining the computational advantages of small mini-batches. The decaying step-size schedule enables exploratory noisy gradient updates in early iterations, and more stable refinement later ([44] (Li et al. 2014)). Furthermore, maintaining a small, constant mini-batch size avoids adverse optimisation behaviour associated with large-batch stochastic optimisation ([41, 1]).

Computational Complexity Analysis

Computational Complexity of the VR Algorithm Proposed Mini-batch stochastic optimisation enables gradient updates to be approximated using subsets of the diffraction data, thereby reducing both computational cost and memory requirements. We analyse the complexity of the proposed algorithm with respect to dataset size and optimiser, retaining the leading-order terms the respective multiplicative constants. Every constant is either a byte count read from the respective implementation, or a quantity measured by a benchmark script given in the code accompanying the paper. The scaling variables used throughout this analysis are summarised in Table 1.

ParameterDescription
K′K^{\prime}Number of diffraction patterns per mini-batch
KKTotal number of diffraction patterns in the dataset
NobjectN_{\mathrm{object}}Number of object pixels
NprobeN_{\mathrm{probe}}Number of probe pixels
LLNumber of optimisation iterations (epochs)
MMNumber of probe modes in the reconstruction
FFNumber of diffraction patterns per PtyPy data block

Table 1. Definition of parameters used in the asymptotic computational complexity analysis.

The memory footprint of the VR algorithm is dominated by the diffraction data held on the device and by the object together with its optimiser state. In the configuration benchmarked here all KK measured patterns are held in GPU memory, so the resident data scales with the size of the dataset. The mini-batch size enters only through the tensors retained for reverse-mode differentiation, not through the resident data. The peak memory, including data-type dependent constants, is

memVR≈memtorch+cNAdamccomplex64Nparams+cfloat32KNprobe+cBKK+max⁡(cact(M)K′Nprobe,copt(Nobject+MNprobe+KM)).(20)\mathit{mem}_{VR} \approx\mathit{mem}_{\mathrm{torch}} + c_{\mathrm{NAdam}} c_{\mathrm{complex64}} N_{\mathrm{params}} + c_{\mathrm{float32}} K N_{\mathrm{probe}} + c_{\mathrm{BKK}} + \max\left(c_{\mathrm{act}}(M) K^{\prime}N_{\mathrm{probe}}, c_{\mathrm{opt}}\left(N_{\mathrm{object}} + MN_{\mathrm{probe}} + KM\right)\right). \tag*{(20)}

Here, memtorch\mathit{mem}_{\mathrm{torch}} is the fixed overhead of the CUDA/PyTorch runtime, measured as 0.65 GB from the first sample of the memory monitor before any data is loaded. The object, the probe modes and the KK per-pattern scaling factors κk\kappa_k of Eq. (7) are stored as single-precision complex numbers, ccomplex64=8c_{\mathrm{complex64}} = 8 bytes, and NAdam holds four arrays of this size for each of them: the parameter, its gradient and two moment estimates, so that cNAdam=4c_{\mathrm{NAdam}} = 4. Diffraction data is stored as single-precision reals, cfloat32=4c_{\mathrm{float32}} = 4 bytes, and cBK=28c_{\mathrm{BK}} = 28 bytes accounts for per-pattern bookkeeping held on the device: the sampling weight of Eq. (16) (4 bytes), the index vector drawn for each epoch (8 bytes) and the integer scan position (16 bytes).

The term max⁡(cact(M)K′Nprobe,copt(Nobject+MNprobe+KM))\max(c_{\mathrm{act}}(M)K'N_{\mathrm{probe}},c_{\mathrm{opt}}(N_{\mathrm{object}}+MN_{\mathrm{probe}}+KM)) is the transient memory of one reconstruction step. The forward propagation of a mini-batch and its gradient allocate cact(M)c_{\mathrm{act}}(M) bytes per detector pixel of the mini-batch; the optimiser step then allocates a temporary copy of every parameter, copt=8c_{\mathrm{opt}} = 8 bytes per element, after the mini-batch tensors have been released. cact(M)c_{\mathrm{act}}(M) is measured from the maximum GPU memory allocated by the PyTorch allocator during one update step, as the slope against K′NprobeK'N_{\mathrm{probe}} over a sweep of K′K' (Table 2). It is 60 bytes for a single mode, where the peak falls in the evaluation of the Poisson negative log-likelihood, and 16+32M16 + 32M bytes for M≥2M \ge2, where the peak falls in the backward pass through the exit waves of every mode. Eq. (20) counts allocated tensors; the per-process memory reported by the nvidia-smi driver, which § 5.3 measures, also includes what the caching allocator holds in reserve, and so exceeds the count predicted by these scaling equations by a small margin.

Probe modes MM1234
cact(M)c_{\mathrm{act}}(M) (bytes)6080112144

Table 2. Transient memory of one VR reconstruction step, measured from the PyTorch allocator as the slope against K′NprobeK'N_{\mathrm{probe}} over a sweep of K′K' sizes, with the data, parameters and optimiser moments already allocated.

The computational cost of the VR algorithm is dominated by FFT operations in the forward model and its adjoint, with a second contribution from the optimiser, which updates every parameter once per mini-batch step. We write the wall-clock time to convergence as TVR=LtepochT_{\mathrm{VR}} = L t_{\mathrm{epoch}}, with

tepoch≈aVRKMNprobe+⌈KK′⌉(bVRNparams+gVR).(21)t_{\mathrm{epoch}} \approx a_{\mathrm{VR}}KMN_{\mathrm{probe}} + \left\lceil\frac{K}{K'}\right\rceil(b_{\mathrm{VR}}N_{\mathrm{params}} + g_{\mathrm{VR}}). \tag*{(21)}

Although mini-batches of size K′K' are used for individual updates, we consider one iteration of the algorithm to be one epoch of the mini-batch method. One epoch is a single pass through all KK patterns, so a mini-batch method with batch size K′K' performs ⌈KK′⌉\left\lceil\frac{K}{K'}\right\rceil parameter-update steps per epoch, with the final mini-batch containing K−K′⌊KK′⌋K - K'\left\lfloor\frac{K}{K'}\right\rfloor patterns when KK is not an integer multiple of K′K'. Typically, multiple epochs are run so that the full dataset is revisited several times, and reporting progress in epochs rather than raw iterations is standard in stochastic-gradient training and gives a more comparable measure across methods with different mini-batch sizes [3].

The first term is the cost of propagating KK patterns through MM modes. The constant aVRa_{\mathrm{VR}} is the time per detector pixel per mode per pass and absorbs the forward and inverse Fourier transforms of the exit waves, the element-wise operations of the forward model and the likelihood, and their adjoints. aVRa_{\mathrm{VR}} is measured as the slope of the step time against K′MNprobeK^{\prime}MN_{\mathrm{probe}} over a sweep of the mini-batch size at fixed object, along which the other two terms of Eq. (21) are constant (Table 3).

ConstantValueStd. errorUnit
aVRa_{\mathrm{VR}}2.48×10−102.48 \times10^{-10}9.0×10−139.0 \times10^{-13}s per K′MNprobeK'MN_{\mathrm{probe}} element
bVRb_{\mathrm{VR}}5.84×10−115.84 \times10^{-11}9.7×10−139.7 \times10^{-13}s per parameter element
gVRg_{\mathrm{VR}}4.45×10−44.45 \times10^{-4}4.9×10−54.9 \times10^{-5}s per step
aMLa_{\mathrm{ML}}2.85×10−102.85 \times10^{-10}2.4×10−122.4 \times10^{-12}s per KMNprobeKMN_{\mathrm{probe}} element
gMLg_{\mathrm{ML}}3.37×10−33.37 \times10^{-3}4.3×10−34.3 \times10^{-3}s per iteration

Table 3. Time constants of Eqs. 21 and 23, measured by benchmarks on the NVIDIA H100 used for every reconstruction, for 128×128128 \times128 diffraction patterns and a single probe mode. Each constant is the slope or intercept of one sweep in which only the corresponding quantity changes.

The second term accounts for the optimiser. NAdam updates the whole parameter set at every mini-batch step, and an epoch contains ⌈K/K′⌉\lceil K/K^{\prime}\rceil such steps, so the optimiser cost per epoch grows as the mini-batch size is decreased. bVRb_{\mathrm{VR}} is measured as the slope of the step time against NparamsN_{\mathrm{params}} over a sweep of the object size at fixed mini-batch.

Computational Complexity of the Maximum-Likelihood Algorithm As a baseline, we consider the maximum-likelihood (ML) algorithm implemented in PtyPy, which employs full-batch first-order gradient updates under a Gaussian statistical noise model. PtyPy subdivides the diffraction data into blocks of FF patterns and manages device residency block by block, so the dominant memory contributions are the resident blocks and the working buffers allocated for a single block. The peak memory, including constant factors, is

memML≈memCuPy+cCGccomplex64(Nobject+MNprobe)+2cfloat32⌈KF⌉FNprobe+(3ccomplex64M+3cfloat32)FNprobe.(22)mem_{\mathrm{ML}} \approx mem_{\mathrm{CuPy}} + c_{\mathrm{CG}}c_{\mathrm{complex64}}(N_{\mathrm{object}} + MN_{\mathrm{probe}}) + 2c_{\mathrm{float32}}\left\lceil\frac{K}{F}\right\rceil FN_{\mathrm{probe}} + (3c_{\mathrm{complex64}}M + 3c_{\mathrm{float32}})FN_{\mathrm{probe}}. \tag*{(22)}

Here, cCG=4c_{\mathrm{CG}} = 4 accounts for storage of the object and probe together with the gradient, the trial gradient and the conjugate search direction. Both algorithms are run in single precision, so the same data-type constants ccomplex64=8c_{\mathrm{complex64} = 8} bytes and cfloat32=4c_{\mathrm{float32}} = 4 bytes apply, giving memMLmem_{\mathrm{ML}} as an estimate of memory usage in bytes. The factor of two on the resident data accounts for the measured intensities and the per-pattern weights, both stored as single-precision reals. As only whole blocks are allocated, the resident data rounds up to a block boundary and exceeds 2cfloat32KNprobe2c_{\mathrm{float32}}KN_{\mathrm{probe}} by up to one block. The final term is the working set of a single block: three complex buffers used by the propagation and gradient kernels, and three real buffers holding the modelled intensity and the likelihood accumulators. It is independent of KK, and for the smallest dataset of § 5.3 it is the largest single contribution to the footprint. memCuPymem_{\mathrm{CuPy}} is the memory overhead associated with the CuPy context, measured as 0.55 GiB.

The time of one ML iteration is likewise dominated by the propagation of every pattern, here performed three times: for the gradient, for the trial step of the polynomial line search and for the update. We write TML=LtiterT_{\mathrm{ML}} = L t_{\mathrm{iter}}, with

titer≈aMLKMNprobe+gML.(23)t_{\mathrm{iter}} \approx a_{\mathrm{ML}}KMN_{\mathrm{probe}} + g_{\mathrm{ML}}. \tag*{(23)}

The constant aMLa_{\mathrm{ML}} is the time per detector pixel per mode per iteration, which was measured as the slope of the iteration time against KMNprobeKMN_{\mathrm{probe}} over seven synthetic datasets of 1.5×1031.5 \times10^{3} to 2.5×1052.5 \times10^{5} patterns, with the parameter tree and block size of the reconstructions of § 5.3.

The ML algorithm performs a single full-batch update per iteration, so the latency gMLg_{\mathrm{ML}} is incurred once per iteration rather than once per diffraction pattern; it is the intercept of the same sweep and is consistent with zero.

Table 3: Time constants of Eqs. (21) and (23), measured by benchmarks on the NVIDIA H100 used for every reconstruction, for 128×128128 \times128 diffraction patterns and a single probe mode. Each constant is the slope or intercept of one sweep in which only the corresponding quantity changes.

Comparison of Computational Complexities The VR algorithm operates on mini-batches of size K′K', whereas the ML algorithm evaluates gradients using the full dataset of size KK at every update. Comparing Eq. (20) with Eq. (22), the memory advantage of VR arises from three distinct sources. First, VR stores only the measured intensities, whereas PtyPy stores intensities and per-pattern noise weights to calculate the Gaussian negative log likelihood model, a factor of two in bytes per detector pixel. Second, PtyPy allocates whole blocks, so its resident data rounds up to a multiple of FF. Third, the transient working set of VR scales with the mini-batch size K′K' while that of PtyPy scales with the block size FF.

The ML memory expression above assumes that every block is resident on the device. PtyPy relaxes this by capping the number of resident blocks at what fits in free memory and streaming the remainder between host and device, so that full-batch ML can run on hardware that cannot hold the entire dataset. Data blocking does not alter the asymptotic comparison with VR, as the number of patterns processed per epoch, and hence the FFT count and floating-point cost, is unchanged. It does, however, introduce repeated host-to-device transfers whose bandwidth and latency overhead scales with KK and is not captured by gMLg_{\mathrm{ML}}.

The time per iteration of both methods is dominated by the FFT in the ptychography model model and therefore exhibits the same dependence on KK, MM and NprobeN_{\mathrm{probe}}. The per-pixel time constants, measured per mode for one ML iteration and one VR epoch respectively, are similar: aMLa_{\mathrm{ML}} is approximately 15% larger than aVRa_{\mathrm{VR}} (Table 3). The effective computational cost per parameter update differs by a factor of approximately K/K′K/K', because ML processes all diffraction patterns at each update whereas VR processes only a subset. On an epoch basis, however, VR is not the cheaper method: it performs a comparable number of propagations per pass over the data, and additionally sweeps the parameter set ⌈K/K′⌉\lceil K/K' \rceil times.

Similar Work

Several recent works address related challenges in ptychographic reconstruction, particularly in computational efficiency, scalability, and modelling flexibility. We briefly review prominent examples in the remainder of this section.

Wu et al. propose an automatic-differentiation framework for low-dose ptychography (DAP) (Wu et al., 2024). DAP minimises a Poisson or Gaussian negative log-likelihood with Adam and reduces gradient noise by increasing the mini-batch size over the course of the reconstruction alongside a plateau-based learning-rate schedule. It is evaluated on simulated data and on measured datasets of up to 4×1044 \times10^{4} diffraction patterns against ePIE and ADMM. The present work differs in three respects. First, we hold the mini-batch size fixed, so peak memory is set by K′K^{\prime} rather than growing during the reconstruction. Second, wall-clock time and peak GPU memory are measured against the full-batch ML algorithm of PtyPy and are predicted by the time and memory models of § 2.6. Third, the evaluation extends to 2×1052 \times10^{5} experimental and 1.02×1061.02 \times10^{6} synthetic diffraction patterns.

Nashed et al. introduce a multi-GPU framework for ptychography [31] and study its scaling behaviour across multiple GPUs using synthetic data. Our work differs from the multi-GPU framework of Nashed et al. in several key aspects: we directly compare against a state-of-the-art ptychography package (PtyPy) on experimental datasets; we demonstrate performance down to a single GPU (rather than a minimum of four GPUs for synthetic and 32 GPUs for experimental data in [31]); and we evaluate a range of experimental conditions with varying memory footprints, frame counts, and probe/object sizes. Furthermore, Nashed et al. use in their simulation a synthetic dataset of 2.56×1042.56 \times10^{4} diffraction patterns, 128×128128 \times128 pixels per diffraction pattern. This is an order of magnitude smaller than the largest synthetic dataset which we consider in § 5.3 of size 128×128128 \times128 pixels per diffraction pattern, 1.02×1061.02 \times10^{6} diffraction patterns. Additionally, the experimental dataset used by Nashed et al. of 4.2×1054.2 \times10^{5} diffraction patterns of size 128×128128 \times128 pixels is smaller than the largest experimental dataset considered in this work as detailed in Table 4. In addition, we explicitly model probe instability.

DatasetNumber of diffraction patternsDiffraction pattern pixel dimensionsObject size pixel dimensions
Nanoprobe Test Target200,000128×128128 \times1281578×19411578 \times1941
Au Particles900512×512512 \times5121095×11031095 \times1103
Butterfly Scale (large)17,496512×512512 \times5126233×28426233 \times2842
Butterfly Scale (small)2346512×512512 \times5121638×18061638 \times1806

Table 4. Experimental datasets used for algorithm validation.

Ghosh et al. propose an automatic-differentiation-based ptychographic reconstruction framework [15]. The automatic-differentiation framework of Ghosh et al. does not consider variance-reduced optimisation, nor does it analyse scaling with problem size or evaluate performance across diverse experimental conditions. Our method, by contrast, combines a differentiable forward model with variance reduction and an explicit Poisson noise model, and includes multi-modal probe modelling to account for partial coherence and probe fluctuations.

Du et al. present Adorym, a general framework for x-ray imaging reconstruction (Du et al., 2021). While it supports flexible forward models, it does not incorporate variance reduction and relies primarily on a single experimental dataset for 2D ptychography, without systematic comparison to other reconstruction algorithms. Our work focuses specifically on variance-reduced optimisation in ptychography and provides direct comparisons to established methods.

Lee et al. introduce PtyRAD, which integrates automatic differentiation into electron ptychography reconstruction (Lee et al., 2025). The PtyRAD reconstruction approach similarly does not incorporate dynamic variance reduction. In contrast, our method applies dynamic variance reduction to x-ray ptychography, analyses computational complexity, and is benchmarked across multiple experimental and synthetic datasets.

Experimental Setup

In this section, we evaluate the proposed variance-reduction (VR) reconstruction algorithm in terms of reconstruction fidelity, computational efficiency, and scaling behaviour.

Reconstruction fidelity is assessed on experimental datasets using the Gaussian negative log-likelihood described in § 2.2, together with visual inspection of object reconstructions. The Gaussian negative log-likelihood serves as the common metric across all algorithms: for ePIE, DM, and ML it is obtained from the per-iteration error values recorded by the PtyPy package. ML minimises the Gaussian NLL, while DM and ePIE are set-projection algorithms that enforce a Fourier-magnitude constraint rather than minimising a statistical likelihood. VR minimises the Poisson negative log-likelihood of Eq. (8), rather than the Gaussian NLL used here as a common reporting metric. A higher Gaussian NLL for VR therefore does not necessarily indicate poorer optimisation, but reflects the differing objectives of the reconstruction methods; correspondingly, VR may attain a lower Poisson NLL than other algorithms. Computational efficiency is quantified by the cumulative runtime required to satisfy the convergence criteria defined in § 3.1, and by peak GPU memory consumption during the execution of the reconstruction script. We report time to convergence as the cumulative iteration time, excluding data loading, up to the point at which the convergence criterion is satisfied. Peak GPU memory usage is measured over the complete execution of the program, including data loading.

We compare the VR algorithm against Difference Map (DM), ePIE, and Maximum Likelihood (ML) implementations from PtyPy. Particular emphasis is placed on the comparison between ML and VR, because both reconstruction algorithms are formulated using statistical noise models. GPU memory usage is a central focus because reducing memory complexity is a principal motivation for introducing stochastic variance-reduced optimisation in large-scale ptychographic reconstruction.

The VR reconstruction routine is implemented in Python using PyTorch (Paszke et al., 2019), with data handled via h5py [5]. The reconstruction is formulated as the minimisation of the Poisson negative log-likelihood, as defined in Eq. (8), which is the sole optimisation objective used by VR. The sampling distribution for gradient updates is defined as a function of the power of each diffraction pattern, as given in Equation (16). Initialising the object with unit pixel-wise magnitude and small random phase ensures consistent gradient magnitudes across datasets and matches the default strategy used in PtyPy. The minimiser is approximated using the NAdam optimiser, implementing a variant of stochastic gradient descent with adaptive updates. Comparisons are performed against PtyPy [36].

All reconstructions were performed on a Linux-based high-performance computing cluster equipped with a 48-core CPU, 1.5 TB of system memory, and a single NVIDIA H100 GPU with 94 GB of device memory. The VR mini-batch size K′K' was chosen for each dataset as a power of two, between 1.8% and 5.9% of the number of diffraction patterns for the experimental datasets; the values used are listed in § 4. These choices were guided by practical considerations and were found empirically to yield stable convergence behaviour.

Convergence Criteria

Let LP\mathcal{L}_{P} denote the Poisson negative log-likelihood (NLL), and let {Lk}k=0N−1\{\mathcal{L}_{k}\}_{k=0}^{N-1} denote the sequence of loss values evaluated at each iteration (or epoch). Convergence is assessed by monitoring the trend of a smoothed loss curve, calculated using the loss curve reported by each algorithm. Smoothing reduces the influence of small fluctuations around the loss function minimum, thereby providing a more robust estimate of the point of convergence. We refer to this rule as the loss-plateau criterion. It tests the slope of the smoothed loss with respect to the iteration number, not the gradient of the objective with respect to the reconstruction parameters.

The loss sequence is first smoothed using an exponential moving average (EMA),

Sk=αLk+(1−α)Sk−1,(24)S_k = \alpha\mathcal{L}_k + (1-\alpha)S_{k-1}, \tag*{(24)}

where SkS_k denotes the smoothed loss, S0=L0S_0 = \mathcal{L}_0, and α=0.095\alpha= 0.095. This value corresponds to an effective averaging window of approximately twenty epochs.

For each epoch k≥19k \ge19, a sliding window comprising of the twenty most recent smoothed loss values is defined as

Wk=Sk−19,…,Sk.(25)W_k = S_{k-19}, \ldots, S_k. \tag*{(25)}

The local trend of the smoothed loss within each window is quantified by fitting a first-order polynomial to the values in WkW_k using ordinary least-squares regression. Let mkm_k denote the resulting slope coefficient, which represents the average rate of change of the smoothed loss over the preceding twenty epochs. The convergence metric is then defined as

Ck=∣mk∣.(26)C_k = \lvert m_k \rvert. \tag*{(26)}

To account for scale variations in the loss function, a normalized slope metric is also computed,

C~k=Ck∣120∑j=k−19kSj∣+ε.(27)\widetilde{C}_k = \frac{C_k}{\left|\frac{1}{20}\sum_{j=k-19}^{k} S_j\right|+\varepsilon}. \tag*{(27)}

where ε=10−12\varepsilon= 10^{-12} ensures numerical stability.

Convergence is declared at the first iteration k∗k^* such that

Ck<τabsorC~k<τrel,(28)C_k < \tau_{\mathrm{abs}} \quad\text{or} \quad\widetilde{C}_k < \tau_{\mathrm{rel}}, \tag*{(28)}

with thresholds τabs=10−4\tau_{\mathrm{abs}} = 10^{-4} and τrel=10−5\tau_{\mathrm{rel}} = 10^{-5}. The threshold values were selected empirically based on preliminary experiments spanning a representative range of datasets and initial conditions. The final values were chosen to provide stable convergence detection across all tested cases. If no such k∗k^* is identified, the algorithm proceeds until reaching the maximum number of pre-specified iterations. In all experiments, the pre-determined maximum number of epochs was chosen to be sufficiently large such that the loss-plateau criterion of Equation (28) was reached before termination.

Experimental Demonstration

We evaluate reconstruction fidelity and computational efficiency on three experimental datasets acquired under varying imaging conditions; see Table 4. The three experimental datasets span photon-counting and non-photon-counting detector types, as well as a wide range of diffraction-frame counts and object field of views.

Experimental Datasets To assess reconstruction quality and computational performance, three experimental datasets were used, which span a variety of experimental conditions. All datasets were collected at the Diamond Light Source. The first dataset are 200,000 ptychographic diffraction patterns from a tungsten test target collected at the I14 nanoprobe instrument with the photon-counting detector (Merlin Quad, Quantum Detectors, UK) at a photon energy of 12 keV. The second dataset are 900 ptychographic diffraction patterns from a sample with Au nanoparticles deposited on a silicon-nitride membrane collected at the I08-1 instrument with the charge-integrating sCMOS detector (AXIS-SXRF-2020EUV, Axis Photonique Inc.) at a photon energy of 710 eV. The third dataset are 17,496 ptychographic diffraction patterns from a brown cover scale taken from a butterfly of the Lycaenid family collected at the I08-1 instrument with the sCMOS Axis Photonique detector at a photon energy of 1 keV. Furthermore, a smaller butterfly scale dataset is used which is cropped to a subset of 2,346 diffraction pattern. In addition to the collection of diffraction patterns, each dataset also includes corresponding probe positions, and other experimental parameters that are required for ptychographic reconstructions. A full list of all the datasets is provided in Table 4 including the size of the reconstructed objects in pixel dimensions.

All diffraction data were stored as 64-bit integer arrays. Reconstructions using the VR algorithm were carried out in PyTorch using complex64 tensors.

Reconstruction hyperparameters and model parameters The Au particle dataset was reconstructed with the probe modelled using three modes to ensure convergence, for both ML and VR. The butterfly scale large and butterfly scale small datasets were reconstructed using two probe modes to ensure convergence, for all reconstruction algorithms. Apart from the number of probe modes, which was the same for both algorithms, two VR settings were chosen for each dataset: the mini-batch size K′K' and the factor parameter. All other VR hyperparameters were identical across all experimental and synthetic datasets. For ML, three settings were chosen for each experimental dataset: the block size FF, and whether the PtyPy scale preconditioner and floating intensities were enabled. The factor parameter controls the proportional reduction in the update step size each time the learning-rate scheduler is applied. The PtyPy block sizes were F=3000F = 3000, 900, 300 and 2500 patterns for the nanoprobe test target, Au particles, butterfly scale (large) and butterfly scale (small) datasets respectively. F=6000F = 6000 for every synthetic dataset. The corresponding VR mini-batch sizes were K′=4096K' = 4096, 16, 1024 and 64. Neither FF nor K′K' was tuned to the GPU used, so the reported memory and runtimes reflect these configurations rather than the minimum achievable by either algorithm.

Reconstruction Results Figure 1 compares object phase reconstructions for the butterfly scale small dataset. The top row, panels (a)–(d), shows reconstructions at termination according to the loss-plateau criterion, while the bottom row, panels (e)–(h), shows the corresponding 5 μ\mum ×\times 5 μ\mum subregion indicated by the dashed boxes.

Figure 1. Phase reconstructions of the butterfly scale small specimen. The top row shows the full-field reconstructions obtained using (a) ePIE, (b) DM, (c) ML, and (d) VR. The bottom row shows the corresponding 5 μm×5 μm5\,\mu\mathrm{m} \times5\,\mu\mathrm{m} magnified region for (e) ePIE, (f) DM, (g) ML, and (h) VR. All panels share a common phase colour scale given in radians. Dashed squares indicate the region shown at higher magnification in the bottom row.

At the stopping point of the loss-plateau criterion, all four algorithms recover the structural features of the object, but differ in sharpness, artefact presence, and noise levels. The ePIE (Figure 1(a)) and VR (Figure 1(d)) reconstructions exhibit comparable overall sharpness, whereas the DM (Figure 1(b)) and ML (Figure 1(c)) reconstructions appear comparatively blurred. These differences are more apparent in the magnified regions: ePIE (Figure 1(e)) exhibits repeated spot-like artefacts, whilst ML (Figure 1(g)) retains greater fine-scale detail than DM (Figure 1(f)) but with slightly increased noise. VR (Figure 1(h)) preserves the greatest level of fine-scale detail in the magnified region, although this is accompanied by a higher apparent noise level. Thus, at this stopping point, the reconstructions exhibit a trade-off between preservation of fine-scale structure and noise, with VR retaining the most detail while DM produces the smoothest appearance.

Figure 1 hence demonstrates that under the aforementioned convergence criterion, the VR algorithm can attain results which are consistent with those attainable by existing ptychography reconstruction algorithms.

Figure 2 shows the Gaussian log-likelihood loss curves corresponding to the reconstructions in Figure 1. The ML algorithm achieves the lowest Gaussian negative log-likelihood loss of all reconstruction algorithms considered. The four algorithms minimise different objectives—VR the Poisson NLL, ML the Gaussian NLL, and DM and ePIE a Fourier-magnitude constraint error—so their loss values reflect these differing formulations as well as the quality of the reconstructed object.

Figure 2. Gaussian log-likelihood loss as a function of cumulative runtime for reconstructions of the butterfly scale small dataset using DM, ML, ePIE, and VR. Both axes are shown on logarithmic scales. Solid lines denote iterations up to the stopping point of the loss-plateau criterion, with circular markers indicating the stopping point. VR minimises the Poisson NLL loss, ML minimises the Gaussian NLL, and DM and ePIE minimise a Fourier-magnitude constraint error.

Each algorithm is assessed individually and is run from the same generic initialisation; no algorithm is warm-started from the reconstruction produced by another, and in particular VR is not initialised from a DM or ePIE reconstruction. This is a deliberate choice: applying VR directly to the measured data, without a prior full-batch reconstruction, is what preserves its low peak-memory footprint, and computing such an initialisation would reintroduce the memory cost that VR is designed to avoid. The lower initial loss of DM and ePIE in Figure 2 is likewise a property of the algorithm class rather than of the starting point: set-projection algorithms apply an exact correction to each constraint set at every iteration, whereas the gradient-based methods minimise a loss function under a step-size schedule chosen to promote stable convergence.

The loss curves of Figure 2 are indicative of convergence behaviour; the absolute “converged” loss values are only a partial indicator of the quality of the object reconstruction. The reconstructions shown in Figure 1 indicate that it is not particularly informative to attempt to delineate between converged loss values within the same order of magnitude.

Table 5 summarises the quantitative performance of each algorithm on the butterfly scale small dataset.

AlgorithmGaussian Negative Log-Likelihood LossTime to convergence (s)Peak GPU memory usage (GB)
ePIE38.224195.895.53
DM15.8831.7330.18
ML13.93264.7847.45
VR67.7881.505.20

Table 5. Reconstruction statistics for the cropped butterfly dataset. The terminal loss is evaluated at the point of convergence defined by the loss-plateau criterion. The loss is the Gaussian negative log-likelihood recorded by PtyPy; VR, which optimises the Poisson negative log-likelihood, records it in a separate, untimed run. Peak GPU memory is the memory attributed to the reconstruction process, over a window that spans data loading; the reported time covers the iterations alone. Experiments were run on an NVIDIA H100 with 94 GB of device memory.

The DM algorithm satisfies the loss-plateau criterion in the lowest cumulative runtime, approximately twice as fast as the VR algorithm. In practical workflows however, DM is typically employed as an initialisation strategy for ML refinement ([40]). When this subsequent optimisation stage is accounted for, the effective time to convergence increases substantially, reducing the apparent computational advantage of DM when considered in isolation. VR, by contrast, is applied directly to the raw data with no initialising reconstruction, so its reported runtime and memory figures reflect the complete cost of the method.

Peak GPU memory consumption is reduced from 47.45 GB (ML) to 5.20 GB (VR), a factor of nine relative to ML and of six relative to DM. These factors reflect the configurations compared as well as the algorithms: with a block size of F=2500F = 2500, PtyPy holds all 2346 diffraction patterns in a single block, and Eq. (22) predicts approximately 6.6 GB for ML with FF equal to the VR mini-batch size of K′=64K^{\prime} = 64. ePIE, which updates the object one diffraction pattern at a time, has a footprint comparable to that of VR (5.53 GB) but takes fifty times longer to satisfy the convergence criterion.

Taken together, the reconstruction-quality, loss, runtime, and memory measurements indicate that VR achieves object reconstructions of comparable visual fidelity whilst maintaining close to an order-of-magnitude advantage in GPU memory usage over the full-batch algorithms at the block size used, and a comparatively quick reconstruction time.

Table 6 reports reconstruction statistics for the full experimental datasets using the ML and VR algorithms. Peak GPU memory usage is lower for VR on every dataset, by factors of between 1.5× and 10.8×. The 10.8× memory saving on the Au particles dataset arises because the PtyPy block size of F=900F = 900 equals the number of diffraction patterns. Consequently, a single block holds the entire dataset and memory allocation for ML scales with FF and the three probe modes, whereas the VR working set scales with its mini-batch size of K′=16K^{\prime} = 16 (Eq. (22)).

DatasetGaussian Negative Log-Likelihood LossTime to convergence (s)Peak GPU memory usage (GB)
MLVRMLVRMLVR
Nanoprobe test target5.9019.76744.23165.2928.8718.76
Au particles5.6310.07770.1876.8226.132.42
Butterfly scale (large)15.8283.971328.82563.6280.1250.54

Table 6. Reconstruction statistics for full experimental datasets comparing ML and VR. The terminal loss is evaluated at the point of convergence defined by the loss-plateau criterion. The loss is the Gaussian negative log-likelihood recorded by PtyPy; VR, which optimises the Poisson negative log-likelihood, records it in a separate, untimed run. Peak GPU memory is the memory attributed to the reconstruction process, over a window that spans data loading. Experiments were run on an NVIDIA H100 with 94 GB of device memory.

The summary statistics of Table 6 give similar indications of the relative performance of the VR algorithm relative to the ML algorithm as demonstrated on the butterfly scale small dataset, now demonstrated on larger datasets. VR reduces the reconstruction time and, at the block sizes used, the peak memory usage, which allows VR to be applied to larger problem sizes where memory is a limiting constraint.

Loss Curves Figure 3 shows the Gaussian NLL as a function of cumulative runtime for ML and VR on the (a) nanoprobe test target, (b) Au particles and (c) butterfly scale (large) datasets, together with the Poisson NLL that VR minimises (right-hand axis). VR satisfies the convergence criterion first on all three datasets. Its terminal Gaussian NLL is higher than that of ML, by factors of 3.3, 1.8 and 5.3 (Table 6), consistent with the different objectives of the two algorithms (§ 3). On the Au particles and butterfly scale (large) datasets the Gaussian NLL of VR shows large transient spikes early in the reconstruction, reaching 10610^{6}–10710^{7}, which are much smaller in the Poisson NLL and do not persist.

Figure 3. Loss as a function of cumulative runtime for ML and VR on the (a) nanoprobe test target, (b) Au particles and (c) butterfly scale (large) datasets. The left axis shows the Gaussian NLL, the common reporting metric; for VR it is recorded in a separate, untimed run. The right axis shows the Poisson NLL that VR minimises. Circles mark the point at which the convergence criterion is satisfied. Both axes are logarithmic.

Object Reconstructions Figure 4 shows the butterfly scale large object reconstructions for the VR and ML algorithms at the convergence point.

Figure 4. Phase reconstructions of the butterfly scale large specimen obtained using (a) ML and (b) VR at the point at which the loss-plateau criterion is satisfied. All panels share a common phase colour scale (rad). A spatial scale bar is shown in panel (a).

Figure 5 presents the Au particles object reconstructions obtained with ML and VR at the point at which the loss-plateau criterion is satisfied.

Figure 5. Phase reconstructions of the Au particles specimen obtained using (a) ML and (b) VR at the point at which the loss-plateau criterion is satisfied. All panels share a common phase colour scale (rad). A spatial scale bar is shown in panel (a).

Figure 6 shows the nanoprobe test target object reconstructions at convergence for ML and VR at the point of loss gradient convergence.

Figure 6. Phase reconstructions of the nanoprobe test target specimen obtained using (a) ML and (b) VR at the point at which the loss-plateau criterion is satisfied. All panels share a common phase colour scale (rad). A spatial scale bar is shown in panel (a).

On the butterfly scale (large) dataset the ML and VR reconstructions are visually indistinguishable at the scale shown (Figure 4). On the Au particles dataset VR resolves more small, high-contrast features than ML and shows more high-frequency texture (Figure 5), the same trade-off between fine-scale detail and noise seen on the butterfly scale (small) dataset. On the nanoprobe test target both algorithms resolve the line-pair groups up to approximately group 8 (Figure 6).

Probe Reconstructions Figure 7 shows the reconstructed probe amplitude and phase for the butterfly scale large dataset at convergence for ML and VR. As the probe was modelled as multimodal for this dataset, the images shown in Figure 7 correspond to the leading singular mode obtained from the singular value decomposition of the reconstructed probe for each algorithm.

Reconstructed probe amplitude and phase for the butterfly scale large dataset, showing ML and VR panels with a colour key

Figure 7. Reconstructed probe amplitude and phase for the butterfly scale large dataset obtained using (a) ML and (b) VR at the point at which the loss-plateau criterion is satisfied. Amplitude is encoded as brightness and phase as hue, according to the colour key.

Figure 8 shows the reconstructed probe amplitude and phase for the Au particles dataset. As the probe was modelled as multimodal for this dataset, the images shown in Figure 8 correspond to the leading singular mode obtained from the singular value decomposition of the reconstructed probe for each algorithm.

Figure 8. Reconstructed probe amplitude and phase for the Au particles dataset obtained using (a) ML and (b) VR at the point at which the loss-plateau criterion is satisfied. Amplitude is encoded as brightness and phase as hue, according to the colour key.

Figure 9 shows the reconstructed probe amplitude and phase for the nanoprobe test target dataset.

Figure 9. Reconstructed probe amplitude and phase for the nanoprobe test target dataset obtained using (a) ML and (b) VR at the point at which the loss-plateau criterion is satisfied. Amplitude is encoded as brightness and phase as hue, according to the colour key.

The ML and VR probes show the same structure on all three datasets but are translated relative to one another. Given that a common translation of object and probe leaves the diffraction data unchanged [46], a shift in the probe reconstruction does not indicate a difference in reconstruction quality.

Visual Reconstruction Quality of the VR Algorithm Across the experimental benchmarks the variance-reduction (VR) algorithm produces object-phase reconstructions that are visually consistent with those from established methods (DM, ePIE, ML). On the butterfly scale small example, VR, DM, and ML recover comparable structural content at the stopping point of the loss-plateau criterion, whereas vanilla ePIE exhibits spatially correlated phase artefacts that are not observed in VR, DM, or ML. Overall, reconstruction fidelity is at least at parity with ML, with fewer visible artefacts than vanilla ePIE and without the mild blurring observed in the DM and ML object reconstructions.

Convergence Behaviour of the VR Algorithm Empirically, VR satisfies the loss-plateau criterion in less wall-clock time than ML. VR uses a single-stage optimisation strategy that minimises the Poisson NLL throughout; unlike ML, which minimises the Gaussian NLL.

Numerical Evaluation of Computational Complexity

This section validates the computational complexity analysis of § 2.6 through numerical experiments. We first describe how the synthetic ptychographic datasets are generated: the probe positions (§ 5.1), the forward model, and the object construction (§ 5.2). We then describe the benchmarks which measure the constants of the time expressions, and compare the peak GPU memory usage and time per pass of the VR and ML algorithms against the expressions of § 2.6.

Generating Probe Positions for Synthetic Datasets

Probe positions on a regular grid induce ill-posedness and lead to reconstruction artefacts [19]. Probe positions for the synthetic datasets are generated on a nominally uniform grid with added Gaussian distributed random displacements to avoid raster-grid artefacts [19]. For a desired overlap ratio and probe diameter, the nominal step size between adjacent positions is computed using the expressions proposed by Moshtaghpour and Kirkland [29].

Synthetic Dataset Construction

The forward model used to generate synthetic data is described by Eq. (29)

Ik=Po(κk∣F[P(r−rk)⋅O]∣2),(29)I_k = P_o\left(\kappa_k \left\lvert\mathcal{F}\left[P(\mathbf{r}-\mathbf{r}_k) \cdot O\right]\right\rvert^2\right), \tag*{(29)}

where Po(⋅)P_o(\cdot) is applied element-wise, with each output sampled from a Poisson distribution whose mean is given by the argument. The propagation operator is approximated by the discrete Fourier transform, F\mathcal{F}. The parameter κk∈R+\kappa_k \in\mathbb{R}^{+} is a scalar intensity-scaling factor used to simulate probe variation between diffraction patterns, as commonly observed in experimental acquisitions.

The probe is simulated as a disc in both phase and amplitude, with added coma, astigmatism, and defocus. The model probe used during reconstruction is identical to the probe used to generate the data, up to a small random perturbation in the added aberration terms. The probe used to generate the synthetic dataset is shown in Figure 10.

Amplitude (left) and phase (right) of the complex-valued probe

Figure 10. Amplitude (left) and phase (right) of the complex-valued probe used to generate the synthetic ptychographic datasets.

Similar to other simulated ptychography demonstrations ([27, 32]), the synthetic object is constructed from two greyscale stock images of size 512×512512 \times512 pixels drawn from the skimage Python library ([52]). The astronaut image defines the object phase, while the camera image defines the object magnitude. The complex-valued object formed from the astronaut phase image and the camera magnitude image is shown in Figure 11.

Amplitude (left) and phase (right) of the complex-valued synthetic object

Figure 11. Amplitude (left) and phase (right) of the complex-valued synthetic object used to generate the diffraction data for the scaling experiments.

To generate larger datasets with the same overlap ratio, the 512×512512 \times512 base object is tiled in each dimension. This ensures that larger datasets consist of repeated structure. Holding the underlying object fixed while varying only the number of scan positions and diffraction patterns isolates the effect of dataset size on scaling behaviour.

To empirically validate the memory and time expressions of § 2.6, synthetic datasets are constructed in which only the number of diffraction patterns is varied. The dimensions of these synthetic datasets are given in Table 7. All diffraction patterns have dimensions 128×128128 \times128 pixels.

Number of diffraction patternsObject sizeMini-batch size K′K'
1.52×1031.52 \times10^{3}511×510511 \times5106464
1.99×1041.99 \times10^{4}1,530×1,5301{,}530 \times1{,}5301,0241{,}024
1.20×1051.20 \times10^{5}3,581×3,5813{,}581 \times3{,}5814,0964{,}096
1.02×1061.02 \times10^{6}10,240×10,24010{,}240 \times10{,}24016,38416{,}384

Table 7. Synthetic datasets used for computational scaling validation. The number of diffraction patterns and reconstructed object grid size are increased systematically to assess algorithmic scaling behaviour. The mini-batch size K′K' of each VR reconstruction is stated, the corresponding PtyPy block size is F=6,000F = 6{,}000 patterns for every dataset.

Scaling Results

Measurement of the Time Constants The constants of Eqs. (21) and (23) are obtained from benchmarks that time single reconstruction steps of the code, optimiser settings and PtyPy parameter tree used in the reconstructions below. For VR, a sweep of the mini-batch size from 6464 to 16,38416{,}384 patterns at a fixed 3,581×3,5813{,}581 \times3{,}581 object gives aVRa_{\mathrm{VR}} as the slope of the step time against K′NprobeK'N_{\mathrm{probe}}; a sweep of the object dimension from 511511 to 10,24010{,}240 pixels at a fixed mini-batch of 1,0241{,}024 patterns gives bVRb_{\mathrm{VR}} as the slope against NparamsN_{\mathrm{params}}; and gVRg_{\mathrm{VR}} is the intercept of either sweep. For ML, individual iterations are timed on seven synthetic datasets of between 1.5×1031.5 \times10^{3} and 2.5×1052.5 \times10^{5} patterns, generated as in § 5.2, with aMLa_{\mathrm{ML}} and gMLg_{\mathrm{ML}} the slope and intercept against KNprobeKN_{\mathrm{probe}}. Each step is timed after warm-up with device synchronisation, and the median of 20 steps (VR) or 50 iterations (ML) is used.

Synthetic Object Reconstructions Figure 12 shows the object phase reconstructions obtained with ML and VR for each of the synthetic datasets of Table 7. Both algorithms recover the tiled object at every dataset size at which they can be executed, so the scaling measurements that follow are obtained from reconstructions of comparable structural content. For the two largest datasets on which ML is feasible, the ML reconstructions show phase offsets between tiles of the object that are large enough for the recovered phase to wrap; the VR reconstructions remain within a single phase branch at all four dataset sizes.

Figure 12. Phase reconstructions of the synthetic object obtained using ML, panels (a)–(d), and VR, panels (e)–(h), at the point at which the loss-plateau criterion is satisfied. Each column corresponds to one of the synthetic datasets of Table 7, containing (a, e) 1.52×1031.52 \times10^{3}, (b, f) 1.99×1041.99 \times10^{4}, (c, g) 1.20×1051.20 \times10^{5}, and (d, h) 1.02×1061.02 \times10^{6} diffraction patterns. Panel (d) is marked N/A because the ML algorithm was not run on the largest dataset, for the reason given in § 5.3. Neither the constant phase offset nor the linear phase ramp of a reconstruction is determined by the data; both are removed from a panel where doing so reduces the wrapping of the displayed phase, and reconstructions that do not wrap are displayed as reconstructed. All panels share a common phase colour scale given in radians.

Table 8 shows the time per iteration over the full dataset, the number of iterations to convergence and the resulting time to convergence for both ML and VR across increasing numbers of diffraction patterns, with the time per iteration compared against the values predicted by Eqs. (21) and (23).

Number of patternsMLVR
titert_{\mathrm{iter}} (s) measuredtitert_{\mathrm{iter}} (s) predictedLLTT (s) measuredtepocht_{\mathrm{epoch}} (s) measuredtepocht_{\mathrm{epoch}} (s) predictedLLTT (s) measured
1.52×1031.52 \times10^{3}0.00800.0105 (+30.6%)585547.00.02410.0173 (−28.3%)2666.4
1.99×1041.99 \times10^{4}0.1040.0963 (−7.8%)116301214.70.09230.0926 (+0.3%)31529.1
1.20×1051.20 \times10^{5}0.5750.563 (−2.0%)111696418.40.5380.523 (−2.8%)338181.9
1.02×1061.02 \times10^{6}N/AN/AN/AN/A4.7064.587 (−2.5%)4051906.0

Table 8. Time per pass over the data for the synthetic datasets: one full-batch iteration for ML and one epoch for VR. Measured values are the reported time to convergence divided by the number of passes LL; predicted values are Eqs. (21) and (23) with the constants of Table 3.

Table 9 shows the peak GPU memory, paired with the byte counts of Eq. (20) and Eq. (22). Convergence is defined using the gradient-based criterion of § 3.1. Peak GPU memory is the memory usage predicted during iterating each reconstruction algorithm, not accounting for any allocations of ML to GPU during data loading. For all dataset sizes where both methods were run, VR reduces peak GPU memory usage and time to convergence relative to ML.

Number of diffraction patternsML peak memory (GB) measuredML peak memory (GB) predictedVR peak memory (GB) measuredVR peak memory (GB) predicted
1.52×1031.52 \times10^{3}1.681.66 (−1.5%)0.940.82 (−12.6%)
1.99×1041.99 \times10^{4}7.337.31 (−0.3%)3.273.04 (−7.1%)
1.20×1051.20 \times10^{5}20.3020.23 (−0.4%)13.6412.95 (−5.1%)
1.02×1061.02 \times10^{6}N/A141.9288.9487.33 (−1.8%)

Table 9. Peak GPU memory of the synthetic reconstructions, measured per process by NVML over a window that spans data loading and the reconstruction, against the byte counts of Eqs. (20) and (22).

‘N/A’ indicates that the ML algorithm was not run on the dataset containing 1.02×1061.02 \times10^{6} diffraction patterns. Eq. (22) gives a resident requirement for ML of 134 GB for the diffraction data alone at the block size used here, and a predicted total of 142 GB, which exceeds the memory of a single Nvidia H100.

The benchmark constants predict the time per pass over the data in Table 8 to within 8% for every run containing 1.99×1041.99 \times10^{4} or more patterns, a factor-of-50 range in KK, and to within 3% for both algorithms on the two largest datasets. The largest error in this range is the ML prediction at 1.99×1041.99 \times10^{4} patterns, which is 7.8% below the measured value. On the smallest dataset the errors are larger, +31% for ML and −28% for VR. This variability therefore affects the VR time to convergence, but not the time per epoch.

The byte count predicted by Eq. (22) agrees with the measured ML peak to within 1.5% at all three dataset sizes. Equation (20) under-predicts the measured VR peak by 2%–13%, with the discrepancy decreasing as the dataset size increases. Overall, the measured improvement factors are 7–42× in time to convergence and 1.5–2.2× in peak GPU memory.

The memory advantage of VR arises from the three effects identified in § 2.6: VR stores only the measured intensities, whereas PtyPy stores both intensities and per-pattern weights; PtyPy rounds its resident data allocation up to a whole number of blocks; and the transient working set of VR scales with K′K', rather than with the block size FF.

Conclusion

Our method applies mini-batch stochastic optimisation with a decaying step size to the Poisson maximum-likelihood formulation of ptychography ([48]). On the datasets tested, it produces reconstructions visually comparable to those of DM, ePIE and ML, and it satisfies the convergence criterion faster than ML with lower peak GPU memory at the configurations compared.

The transient memory of VR is set by its mini-batch size K′K^\prime, and that of PtyPy ML by its block size FF; both are user-selected parameters, so measured memory ratios depend on the configurations compared. At the settings used in this work, peak GPU memory for VR is lower than for ML on every experimental dataset, by factors of between 1.5 and 10.8. The largest reductions, from 47.45 GB to 5.20 GB on the butterfly scale (small) dataset and a factor of 10.8 on the Au particles dataset, arise where ML was run with blocks far larger than the VR mini-batch (F=2500F = 2500 against K′=64K^\prime= 64, and F=900F = 900 against K′=16K^\prime= 16), so that the working set of a single block dominates the ML footprint. Eq. (22) predicts that reducing FF to the VR mini-batch size would lower the ML footprint on these datasets to approximately 6.6 GB and 2.9 GB, within a factor of 1.5 of the corresponding VR predictions; the effect of such small blocks on the ML runtime was not measured. The part of the advantage that is independent of FF and K′K^\prime lies in the resident data: VR stores only the measured intensities, whereas PtyPy additionally stores per-pattern weights, which bounds the memory advantage of VR at a factor of two once the resident data dominates, as for the nanoprobe test target (1.5×). PtyPy can also reduce its device footprint by streaming blocks between host and device, at a transfer cost not captured by Eq. (22); the largest synthetic dataset is therefore infeasible for ML on a single H100 only without such streaming.

Synthetic scaling experiments, in which only the number of diffraction patterns is varied, confirm the complexity analysis: with constants measured independently of the reconstructions, the time per pass is predicted to within 8% for every run of more than 10410^4 patterns, and to within 3% on the two largest datasets, the ML peak memory to within 1.5%, and the VR peak memory to within 13%, over three orders of magnitude in frame count. Peak memory grows linearly with frame count for both algorithms, but with half the slope for VR, and the time to convergence of VR grows more slowly than that of ML.

Taken together, these results show that VR achieves a lower memory footprint than PtyPy ML at the configurations tested, with an advantage of up to a factor of two in resident data that persists for any block size, and favourable computational scaling for large-scale ptychographic reconstruction.

From a broader computational imaging perspective, this work illustrates the practical value of approximation techniques from the machine learning literature—such as stochastic gradient descent and variance reduction—when applied to large-scale inverse problems. While SGD and its variants only approximate the full gradient, on the datasets tested the associated approximation error does not visibly degrade the reconstruction relative to full-batch ML. Whether mini-batch gradient noise also acts as an implicit regulariser in ptychography, as has been argued for deep learning [18], is not tested here. The stochastic gradient noise introduced by mini-batch sampling can, in turn, be tuned via the mini-batch size.

The present work focuses on a Poisson maximum-likelihood formulation of ptychography. Extending variance-reduced optimisation to alternative formulations, including set-projection algorithms, remains an open direction. Since only the mini-batch size and the step-size decay factor were set for each dataset, and since similar algorithms are used across different ptychographic modalities, it is reasonable to expect that the proposed variance-reduced reconstruction framework would transfer with limited modification to other modalities such as electron ptychography.

The argument that variance-reduced optimisation should transfer effectively to related imaging modalities also applies to higher-dimensional imaging, such as ptycho-tomography ([17, 6]), where the dimensionality of the inverse problem is substantially larger. In higher-dimensional imaging settings such as ptycho-tomography, the scaling advantages of stochastic and variance-reduced methods over full-gradient optimisation may be even more pronounced.

Further computational gains may be attainable by employing adaptive optimisers with lower memory requirements than Adam. Examples include MicroAdam ((Modoranu et al., 2024)), a memory-efficient optimiser that compresses gradient information, and Adafactor ([42]), which reduces memory requirements by factorising second-moment estimates. Optimisers such as Shampoo ([16]), which have demonstrated faster convergence than Adam on certain machine learning tasks, could also be considered. In this work, we restrict ourselves to established optimisers available within the PyTorch ecosystem; evaluating alternative optimisers in the context of ptychographic reconstruction, and quantifying their impact on convergence, memory usage, and hyperparameter sensitivity, is left for future work.

An important avenue for improvement is the development of more informative reconstruction quality metrics, which would better indicate both convergence and solution quality ([45]). Designing such metrics is challenging, as ptychography is an ill-posed inverse problem without a unique ground-truth solution.

Finally, there is an opportunity to further explore connections between generalisation in deep learning and robustness in inverse problems. In supervised learning, hyper-parameters such as batch size influence the generalization gap between training and unseen data ([18] (Li et al. 2014)). In ptychography and related inverse problems, analogous hyper-parameters affect the ability of an optimisation algorithm to fit an imperfect forward model to experimental data while avoiding over-fitting to noise or artefacts. Concepts such as small-batch training and implicit regularisation may therefore translate between these domains, which motivates deeper theoretical and empirical study of the relationship between generalization in deep learning and robustness in inverse problems.

Acknowledgements

This work was carried out with the support of Diamond Light Source, instruments I14 (in-house data) and I08-1 (proposal MG36964 and in-house data). The authors would like to thank Dr. Duane Loh, Dr. Anupama Prakash, Mr. Ervin Chia and Mr. Justin Ong for their help and support during the MG36964-1 experiment. The authors would further like to thank Dr. Majid Kazemian for the in-house data collection of the Au nanoparticles dataset. The authors would also like to thank Dr. Miguel Gomez Gonzalez for the in-house data collection of the I14 nanoprobe test target.

Funding

A.W.M. was supported by a Scientific Computing PhD studentship from the Ada Lovelace Centre.

J.T. is supported by the UK Engineering and Physical Sciences Research Council (EPSRC) through the grant EP/Y028872/1; Mathematical Foundations of Intelligence: An “Erlangen Programme” for AI.

Conflicts of interest: There are no conflicts of interest.

Data availability: The Au nanoparticles and nanoprobe test target datasets are both available here: https://doi.org/10.5281/zenodo.11501765. The butterfly scale dataset is available upon reasonable request. The implementation of the variance-reduced reconstruction framework, together with the scripts used to perform all reconstructions and generate all figures and tables reported in this article, will be made openly available at https://github.com/am318/variance-reduction-ptychography after publication.

References

  1. [1]Arnaboldi, L., Dandi, Y., Krzakala, F., Loureiro, B., Pesce, L. & Stephan, L. (2024). In Proceedings of the 41st International Conference on Machine Learning, edited by R. Salakhutdinov, Z. Kolter, K. Heller, A. Weller, N. Oliver, J. Scarlett & F. Berkenkamp, vol. 235 of Proceedings of Machine Learning Research, pp. 1730–1762. PMLR. https://proceedings.mlr.press/v235/arnaboldi24a.html
  2. [2]Barbieri, D., Bonforte, M. & Ibarrondo, P. (2026). Is stochastic gradient descent effective? a pde perspective on machine learning processes. https://arxiv.org/abs/2501.08425
  3. [3]Bottou, L., Curtis, F. E. & Nocedal, J. (2018). SIAM Review, 60(2), 223–311. https://doi.org/10.1137/16M1080173
  4. [4]Chen, B., Xu, Y. & Shrivastava, A. (2019). In Advances in Neural Information Processing Systems, edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox & R. Garnett, vol. 32. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2019/file/a1e865a9b1065392ed6035d8ccd072d9-Paper.pdf
  5. [5]Collette, A. (2013). h5py: Python interface to HDF5. https://www.h5py.org/. https://www.h5py.org/
  6. [6]Dierolf, M., Menzel, A., Thibault, P., Schneider, P., Kewish, C. M., Wepf, R., Bunk, O. & Pfeiffer, F. (2010). Nature, 467(7314), 436–439. https://doi.org/10.1038/nature09419
  7. [7]Dong, Z., Fang, Y., L. Huang, X., Yan, H., Ha, S., Xu, W., Chu, Y. S., Campbell, S. I. & Lin, M. (2018). In 2018 New York Scientific Data Summit (NYSD S), pp. 1–5.
  8. [8]Dozat, T. (2016). In International Conference on Learning Representations (ICLR) Workshop Track.
  9. [11]Dwivedi, P., Konijnenberg, A. P., Pereira, S. F. & Urbach, H. P. (2017). In Optical Measurement Systems for Industrial Inspection X, edited by P. Lehmann, W. Osten & A. A. G. Jr., vol. 10329, p. 10329Y. International Society for Optics and Photonics, SPIE. https://doi.org/10.1117/12.2270165
  10. [12]Enders, B. & Thibault, P. (2016). Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 472(2196), 20160640. https://doi.org/10.1098/rspa.2016.0640
  11. [13]Fannjiang, A. & Strohmer, T. (2020). Acta Numerica, 29, 125–228.
  12. [14]Favre-Nicolin, V., Girard, G., Leake, S., Carnis, J., Chushkin, Y., Kieffer, J., Paleo, P. & Richard, M.-I. (2020). Journal of Applied Crystallography, 53(5), 1404–1413. https://doi.org/10.1107/S1600576720010985
  13. [15]Ghosh, S., Nashed, Y. S. G., Cossairt, O. & Katsaggelos, A. (2018). In 2018 IEEE International Conference on Computational Photography (ICCP), pp. 1–10.
  14. [16]Gupta, V., Koren, T. & Singer, Y. (2018). In Proceedings of the 35th International Conference on Machine Learning, edited by J. Dy & A. Krause, vol. 80 of Proceedings of Machine Learning Research, pp. 1842–1850. PMLR. https://proceedings.mlr.press/v80/gupta18a.htmlDOI
  15. [17]Gürsoy, D. (2017). Opt. Lett. 42(16), 3169–3172. https://opg.optica.org/ol/abstract.cfm?URI=ol-42-16-3169
  16. [18]Hoffer, E., Hubara, I. & Soudry, D. (2017). In Advances in Neural Information Processing Systems, edited by I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan & R. Garnett, vol. 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2017/file/a5e0ff62be0b08456fc7f1e88812af3d-Paper.pdf
  17. [19]Huang, X., Yan, H., Harder, R., Hwu, Y., Robinson, I. K. & Chu, Y. S. (2014). Opt. Express, 22(10), 12634–12644. https://opg.optica.org/oe/abstract.cfm?URI=oe-22-10-12634
  18. [20]Johnson, R. & Zhang, T. (2013). In Advances in Neural Information Processing Systems, edited by C. Burges, L. Bottou, M. Welling, Z. Ghahramani & K. Weinberger, vol. 26. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2013/file/ac1dd209cbcc5e51c6e28598c8cbce8-Paper.pdf
  19. [22]Ko, T. & Li, X. (2023). Journal of Machine Learning, 2(2), 138–160. https://doi.org/10.4208/jml.v2.n2.p138
  20. [23]Konijnenberg, S. (2017). Advanced Optical Technologies, 6.
  21. [26]Luke, D. R. (2004). Inverse Problems, 21(1), 37. https://doi.org/10.1088/0266-5611/21/1/004
  22. [27]Maiden, A. M. & Rodenburg, J. M. (2009). Ultramicroscopy, 109(10), 1256–1262. https://www.sciencedirect.com/science/article/pii/S0304399109001284
  23. [29]Moshtaghpour, A. & Kirkland, A. I. (2025). In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5.
  24. [30]Moshtaghpour, A., Velazco-Torrejon, A., Robinson, A. W., Browning, N. D. & Kirkland, A. I. (2025). Opt. Express, 33(5), 9357–9368. https://opg.optica.org/oe/abstract.cfm?URI=oe-33-5-9357
  25. [31]Nash, Y. S., Peterka, T., Deng, J. & Jacobsen, C. (2017a). Procedia Computer Science, 108, 404–414. International Conference on Computational Science, ICCS 2017, 12-14 June 2017, Zurich, Switzerland. https://www.sciencedirect.com/science/article/pii/S1877050917306531
  26. [32]Nash, Y. S., Peterka, T., Deng, J. & Jacobsen, C. (2017b). Procedia Computer Science, 108, 404–414. International Conference on Computational Science, ICCS 2017, 12-14 June 2017, Zurich, Switzerland. https://www.sciencedirect.com/science/article/pii/S1877050917306531
  27. [33]Odstrčil, M., Menzel, A. & Guizar-Sicairos, M. (2018). Opt. Express, 26(3), 3108–3123. https://opg.optica.org/oe/abstract.cfm?URI=oe-26-3-3108
  28. [34]Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J. & Chintala, S. (2019). In Advances in Neural Information Processing Systems, edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. Fox & R. Garnett, vol. 32. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf
  29. [35]Pham, M., Rana, A., Miao, J. & Osher, S. (2019). Opt. Express, 27(22), 31246–31260. https://opg.optica.org/oe/abstract.cfm?URI=oe-27-22-31246
  30. [36]PtyPy Developers (2025). PtyPy: the ptychography reconstruction framework. Https://ptypy.org/. https://ptypy.org/
  31. [37]Robbins, H. & Monro, S. (1951). The Annals of Mathematical Statistics, 22(3), 400 – 407. https://doi.org/10.1214/aoms/1177729586
  32. [38]Rodenburg, J. (2025). Journal of Microscopy, 300(2), 153–155. https://onlinelibrary.wiley.com/doi/abs/10.1111/jmi.70025
  33. [39]Rodenburg, J. M. & Faulkner, H. M. L. (2004). Applied Physics Letters, 85(20), 4795–4797. https://doi.org/10.1063/1.1823034
  34. [40]Sala, S., Kuppili, V. S. C., Chalkidis, S., Batey, D. J., Shi, X., Rau, C. & Thibault, P. (2018). Journal of Synchrotron Radiation, 25(4), 1214–1221.
  35. [41]Sclocchi, A. & Wÿart, M. (2024). Proceedings of the National Academy of Sciences, 121(9), e2316301121. https://www.pnas.org/doi/abs/10.1073/pnas.2316301121
  36. [42]Shazeer, N. & Stern, M. (2018). In Proceedings of the 35th International Conference on Machine Learning, edited by J. Dy & A. Krause, vol. 80 of Proceedings of Machine Learning Research, pp. 4596–4604. PMLR. https://proceedings.mlr.press/v80/shazeer18a.htmlDOI
  37. [43]Smith, S., Elsen, E. & De, S. (2020). In Proceedings of the 37th International Conference on Machine Learning, edited by H. D. III & A. Singh, vol. 119 of Proceedings of Machine Learning Research, pp. 9058–9067. PMLR. https://proceedings.mlr.press/v119/smith20a.htmlDOI
  38. [44]Smith, S. L., Kindermans, P.-J. & Le, Q. V. (2018). In International Conference on Learning Representations. https://openreview.net/forum?id=B1Yy1BxCZ
  39. [45]Thibault, P. (2024). Algorithm best practices. Talk presented at the PtyPy Workshop 2024, https://ptypy.org/workshop2024/. https://ptypy.org/ptpyworkshop2024/
  40. [46]Thibault, P., Dierolf, M., Bunk, O., Menzel, A. & Pfeiffer, F. (2009). Ultramicroscopy, 109(4), 338–343. https://www.sciencedirect.com/science/article/pii/S0304399108003458
  41. [47]Thibault, P., Dierolf, M., Menzel, A., Bunk, O., David, C. & Pfeiffer, F. (2008). Science, 321(5887), 379–382. https://www.science.org/doi/abs/10.1126/science.1158573
  42. [48]Thibault, P. & Guizar-Sicairos, M. (2012). New Journal of Physics, 14(6), 063004. https://doi.org/10.1088/1367-2630/14/6/063004
  43. [49]Thibault, P. & Menzel, A. (2013). Nature, 494(7435), 68–71. https://doi.org/10.1038/nature11806
  44. [50]Vartanyants, I. A. & Singer, A. (2010). New Journal of Physics, 12(3), 035004. https://doi.org/10.1088/1367-2630/12/3/035004
  45. [52]van der Walt, S., Schönberger, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., Gouillart, E., Yu, T. & the scikit-image contributors (2014). PeerJ, 2, e453.
  46. [53]Wang, X., Magnússon, S. & Johansson, M. (2021). In Advances in Neural Information Processing Systems, edited by M. Ranzato, A. Beygelzimer, Y. Dauphin, P. Liang & J. W. Vaughan, vol. 34, pp. 14226–14238. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2021/file/76c538125fc5c9ec6ad1d05650a57de5-Paper.pdf
  47. [55]Yan, H. (2020). New Journal of Physics, 22(2), 023035. https://doi.org/10.1088/1367-2630/ab704e

Paper details

vlnc:W_n8wfmfvd
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Engineering

Not recorded for this paper

View statement
Looking for related papers…
Contents