betaFeedback

WorksPhysics and Astronomy

Automatic Parameter Selection for Electron Ptychography via Bayesian Optimization

Abstract

Electron ptychography provides new opportunities to resolve atomic structures with deep sub-angstrom spatial resolution and studying electron-beam sensitive materials with high dose efficiency. In practice, obtaining accurate ptychography images requires simultaneously optimizing multiple parameters that are often selected based on trial-and-error, resulting in low-throughput experiments and preventing wider adoption. Here, we develop an automatic parameter selection framework to circumvent this problem using Bayesian optimization with Gaussian processes. With minimal prior knowledge, the workflow efficiently produces ptychographic reconstructions that are superior than the ones processed by experienced experts. The method also facilitates better experimental designs by exploring optimized experimental parameters from simulated data.

Introduction

Ptychography is a computational imaging method that has gained great interests in the electron microscopy community[1, 2, 3, 4]. The technique was first proposed by Hoppe in 1969[5] and re-invigorated in recent years with the developments of fast electron detectors[6, 7, 8] that can rapidly collect tens of thousands of diffraction patterns in less than a minute. Various iterative

Automatic Parameter Selection for Electron Ptychography via BO

reconstruction algorithms have been developed to retrieve the scattering potentials of the sample and the wave function of the illumination from intensity measurements[9, 10, 11]. It has been demonstrated that electron ptychography can break the Abbe diffraction limit of imaging systems[12] and set a new world record in spatial resolution (0.39 Å) in atomically thin two-dimensional (2D) materials[3]. As one of the phase-contrast imaging techniques, electron ptychography also has high dose-efficiency for low-dose imaging ranging from low-dimensional nanomaterials[13, 14] to biological specimens[15, 16]. An even more critical breakthrough is that electron ptychography can inversely solve the long-standing problem of multiple scattering in thick (>20>20 nm) samples and enables a lattice-vibration-limited resolution (0.2 Å)[4], as well as three-dimensional depth sectioning[4, 17].

Despite its great success in achieving record-breaking resolution, ptychography remains a niche technique in electron microscopy due to many practical challenges in both experimental setup and data analysis. In particular, there exist many types of parameters that significantly influence image quality and need to be carefully selected for different data or applications. Because modern ptychography recovers object structures via solving an iterative inverse problem, the recovered information strongly depends on the reconstruction parameters, which are related to physical modeling of the data as well as optimization algorithms. In addition, experimental parameters such as scan step size and probe defocus also limits the best image quality of a given measurement. Due to the virtually infinite possibilities and complex trade-offs between various parameters, it is practically impossible to design and optimize ptychography experiments by searching the entire parameter space. In most literature[3, 4, 13, 14], both reconstruction and experimental parameters are manually selected based on scientists’ experiences about the sample or the instrument. This not only introduces biases to scientific conclusions that are drawn from the reconstruction but greatly reduces the throughput of the overall workflows, resulting in a high barrier for general researchers to adopt the technique.

Here we present a general framework for fully automatic parameter tuning in electron ptychography by leveraging Bayesian optimization (BO) with Gaussian processes[18]—a popular strategy for global optimization of unknown functions. Using experimental ptychography data and state-of-the-art reconstruction algorithms, we demonstrated that our approach can automatically produce high-resolution images after exploring only 1% of the entire reconstruction parameter space. We also optimized experimental parameters for ultra-low electron dose levels, providing insights for more robust experimental designs that further to enhance ptychography’s usability. Instead of relying on human intuition and judgment, automatic parameter selection promotes objective and reproducible protocols, paving the way for fully autonomous experiments and data processing for ptychography applications.

Results

Bayesian Optimization with Gaussian Process

Bayesian optimization with Gaussian process is frequently used to find global maxima and minima of a black-box function that is unknown and expensive to evaluate. The technique has been used in a wide variety of applications in machine learning[19, 20], Monte Carlo simulation[21], and autonomous controls in microscopy experiments[22, 23, 24]. In general, BO consists of three steps: 1) compute a surrogate function that models the true objective function based on sampled points, 2) determine the next point(s) to be sampled based on an acquisition function, 3) evaluate the objective function at the corresponding points. The surrogate function is described by kernel functions, which affect the periodicity, smoothness, and length scales of the objective function. Maximizing the acquisition function provides the points that automatically balance the trade-off between finding the extrema (exploitation) or reducing the uncertainty in the surrogate (exploration).

In ptychography, we utilize BO to optimize an objective function that evaluates reconstruction quality and varies for each dataset. Figure 1 illustrates the complete workflow. The initial set of ptychography reconstructions are generated based on randomly chosen parameters. Based on these reconstructions, Gaussian process models a surrogate function, and candidate points thereafter are chosen according to the acquisition function. The following reconstructions are performed with these parameters, the quality is measured, and the surrogate and acquisition functions are updated. The updated acquisition function then suggests the next candidate set and the process is iterated (Figure 1b). After sufficient iterations, the set of parameters that generate the highest quality reconstruction is determined (Figure 1c).

Schematic of automatic reconstruction tuning with Bayesian optimization

Figure 1. Schematic of automatic reconstruction tuning with Bayesian optimization. (a) The process aims to find the best ptychographic reconstruction by optimizing an unknown quality function that is data-dependent in general. (b) Bayesian optimization loop strategically determines the next point (indicated in orange) to sample, performs ptychographic reconstruction, and then updates the surrogate model based on the image quality. As the number of iterations increases, the surrogate model becomes closer to the true quality function and more points around the optimum are exploited. (c) The image with the best quality during BO is retrieved as the final reconstruction.

Automatic Reconstruction Parameter Tuning

To demonstrate BO as an efficient framework for automatic selection of reconstruction parameters, we apply the approach on an experimental dataset of bilayer MoSe2/WS2\mathrm{MoSe}_{2}/\mathrm{WS}_{2} which is publicly available from ref. [13]. Ptychographic reconstructions were carried out using the least square maximum likelihood (LSQ-ML) algorithm[25] implemented in the PtychoShelves package[26], which incorporates many advanced techniques such as mixed-states ptychography[27], position correction[25], variable probe correction[28], batch update[25], and multislice ptychography[4, 29]. These features play crucial roles in previous works that successfully achieved dose-efficient and large field of view (FOV) imaging[13, 30] as well as the deep sub-angstrom spatial resolution of thick crystalline materials[4]. Because different parameters have different computational costs per iteration, we perform time-limited reconstructions that terminate after reaching a time threshold specified by the user. This provides more practical comparisons by balancing the trade-offs of parameters such as the number of probe modes and batch size. Assuming strong phase approximation, which generally works well for 2D materials, the complete parameter space consists of eight discrete parameters and a total number of 4800 possible combinations. Detailed descriptions of each reconstruction parameter are provided in the Methods section and supplementary Table 1.

Reconstruction ParameterValue
Number of mixed-state modes1, 2, 3, 4, 5, 6, 7, 8, 9, 10
Batch size60, 120, 180, 240, 300
Batch selection schemeSparse, Compact
Number of OPR modes0, 1, 2
Intensity correctionTrue, False
Position correctionTrue, False
Modal updateTrue, False
Noise modelGaussian, Poisson

Table 1. Reconstruction parameters and possible values in automatic parameter tuning. There are a total number of 4800 possible combinations.

Fourier ring correlation (FRC) analysis [31] was used as a quantitative metric to evaluate the quality of ptychographic reconstructions. Without the “ground truth” for experimental data, FRC analysis measures the similarity between two independent reconstructions and is often used to estimate “spatial resolution” in phase retrieval problems [32] or cryogenic electron microscopy reconstructions [33]. Our automatic parameter tuning workflow aims to maximize the area under the normalized FRC curve, which ranges from 0 to 1 with 1 corresponding to identical images. The process starts by trying 5 initial random sets of reconstruction parameters, then leverages BO to search for the next point that is most likely to produce better reconstruction quality, and stops after exploring 50 points in total – only 1% of the entire parameter space. A fully random parameter selection strategy was also investigated as comparison. Each sampling strategy was carried out 10 times (with different initial starts) and the averaged best FRC scores after each search are shown in Figure 2a. It is obvious that BO can consistently achieve higher FRC scores than random sampling, even if it starts with worse averaged FRC scores. The frequency of specific reconstruction parameter values also demonstrates that BO tends to sample more points around the optimal parameters. For example, for all 3-minute reconstructions, the percentage of position correction used in random and BO sampling are 71.2% and 48.4%, respectively. Selected reconstructions from BO and random parameter selections are shown in Figure 2b and Figure 2d, respectively. The reconstructed atoms from BO parameter tuning are visibly sharper than the ones from random sampling, which agrees with the evaluation based on FRC and their diffractograms (Fourier intensity) (Figure 2c&Figure 2e). It is worth noting that the diffraction spot that corresponds to an Abbe

Performance of Bayesian optimization versus a random search

Figure 2. Performance of Bayesian optimization versus a random search. (a) Plot of FRC score behavior over number of parameter searches using BO versus a random search strategy. In 10 trials, BO consistently outperforms random search with higher FRC score and lower uncertainty as the number of parameter searches increases. (b,d) Best reconstruction for one trial after 50 parameter searches using (b) Bayesian optimization and (d) random search. (c,e) Diffractograms of reconstructions in (b) and (d), respectively.

resolution of 0.42 Å is visible in the reconstruction with optimized parameters, surpassing the resolution (0.69 Å) reported in our previous work[13].

Figure 3 illustrates the importance of reconstruction parameter tuning by showing 5-minute reconstructions. The best results (Figure 3a&Figure 3b) found by BO used 7 mixed-states probe modes, a sparse batch size of 300, a Gaussian noise model, and position correction. These parameters agree well with the choice made by experienced scientists who are familiar with the algorithm and the data[13]. For comparison, a smaller batch size of 60 produces reconstructions (Figure 3c&Figure 3d) that have broader atoms and a slightly lower FRC score than the optimal results. Moreover, reconstructions with no position correction and only a single probe mode are shown in Figure 3e&Figure 3f and g&h, respectively. Because the experimental errors are not corrected during reconstruction, these images have significantly worse quality and inconsistencies that are reflected in their FRC scores. The complete list of reconstruction parameters used for Figure 3 is summarized in Supplementary Table 2.

Optimized ptychographic reconstructions of bilayer MoSe2/WS2 compared with sub-optimal parameters

Figure 3. Optimized ptychographic reconstructions of bilayer MoSe2/WS2\mathrm{MoSe}_{2}/\mathrm{WS}_{2} compared with sub-optimal parameters. Two datasets that cover the same scan area were reconstructed independently for 5 minutes using the LSQ-ML technique. (a,b) Reconstructed phase with the parameters optimized by Bayesian optimization, including 7 mixed-state probe modes, a batch size of 300, and scan position correction. (c-f) Reconstructions where one of the optimal parameters is changed. All sub-optimal parameter combinations decrease the reconstruction quality to varying degrees. (c,d) Reconstructions with a batch size of 60, which increases the time per iteration, and hence the total number of iterations is reduced. (e,f) Reconstructions without position correction. (g,h) Reconstructions with a single probe mode.

Using the efficient and automatic parameter tuning enabled by BO, one can gain deeper understandings of optimal reconstruction parameters and systematically study how they change with time. As shown in Figure 4, for the experimental dataset of MoSe2_2/WS2_2, the phase of the object converges to the bilayer structures within 10 minutes, as the probe modes quickly become more physical shapes. As time further increases, the FRC scores continue to improve, which is mainly thanks to scan position correction that often requires more iterations to refine large drifts or global position errors. The plots for individual reconstruction parameters are provided in Supplementary Figure

Ptychographic reconstructions of bilayer MoSe$_2$/WS$_2$ at different time limits

Figure 4. Ptychographic reconstructions of bilayer MoSe2_2/WS2_2 at different time limits. The best reconstruction parameters at each time limit were automatically optimized by BO. (a) Best FRC scores vs. reconstruction time. (b-d) Best reconstructions and their diffractograms (insets) after 3, 10, and 40 minutes, respectively. The zoom-ins in (a) shows that reconstructed atoms become more resolved as time increases.

  1. For all time limits, BO indicates that the best results are obtained with a large number of probe modes, position correction, and Gaussian noise model. It also suggests that no probe variation correction is needed – this is within our expectations since the scan FOV is relatively small (5 nm×5 nm5\ \mathrm{nm} \times5\ \mathrm{nm}). In the early reconstruction stage (<25<25 minutes), the sparse batch selection scheme leads to higher FRC scores since the algorithm has a faster initial convergence rate. On the other hand, given enough time (number of iterations), the algorithm benefits more from the compact batch selection scheme that is known to have slower convergence but is more robust to noise[25]. We further compared the reconstructed atomic distances with the structure model, and confirmed that the compact batch indeed produces more accurate results than the sparse batch at longer time limits.

Optimization of Experimental Parameters for Low-dose Ptychography

The dose of the illumination beam plays a crucial role in electron microscopy. For example, high electron dose can damage the sample structure by energetic electrons, especially for radiation-sensitive samples, such as batteries, metal-organic frameworks, or biological materials[34, 35]. In contrast, low dose mode results in noisy diffraction patterns, reducing spatial resolution or even introducing additional artifacts in ptychographic reconstructions. Therefore, it is critical to explore optimal imaging conditions at the allowed dose of illumination.

In general, experimental conditions, such as scan step size and probe size, often determine the best quality one can achieve after ptychographic reconstruction. The optimal experimental parameters should balance various physical factors such as the signal to noise ratio (SNR) of diffraction patterns, scanning probe overlap, and the sampling requirement in the detector plane[36, 37]. Due to the complex tradeoffs between different factors, it is generally challenging, even for human experts, to determine the optimal parameters that maximize the reconstruction quality in different experiments. For instance, at a fixed electron dose, a small scan step size leads to a large number of diffraction patterns with electron counts collected in the detector. Increasing scan step size could improve the SNR but reduce the spatial overlap between adjacent probes. Although larger probe defocuses could provide better overlap in real space, it requires higher sampling (more pixels) in the detector plane, which again lowers the SNR since the averaged electron count per pixel is decreased.

Using Bayesian optimization with Gaussian processes, we performed a comprehensive and automatic parameter tuning to search for the optimal scan step size, aperture size, probe defocus, and detector size (Supplementary Table 3) at different electron dose levels. For each point in the 4D parameter space, we first simulated diffraction patterns using a twisted bilayer MoS2\mathrm{MoS}_{2} structure (Supplementary Figure S2) as the test object and carried out ptychographic reconstructions using the LSQ-ML algorithm. With the ground truth available,

Projected potentials of simulated bilayer MoS₂ structure with a 30° twist

Figure S2. Projected potentials of simulated bilayer MoS2\mathrm{MoS}_{2} structure with a 30∘30^\circ twist. The object was used to simulate electron ptychograpy data in automatic experimental parameter optimization.

BO directly maximizes the accuracy of reconstruction, which is quantified with the structural similarity index measure (SSIM)[38]. Figure 5a-Figure 5d shows the best reconstructions after 800 points are explored at various dose levels from 100 to 50,000 e−/A˚2e^{-}/\mathring{\mathrm{A}}^{2}. As references, reconstructions with a fixed set of experimental parameters (2 A˚\mathring{\mathrm{A}} scan step size, 20 mrad aperture size, -55 nm defocus, 256 ×\times 256 detector size), which are similar to the ones used in ref. [13], are shown in Figure 5e-Figure 5h. The experimental parameters optimized by BO produce significantly better resolution and more accurate structures, especially at lower dose levels where the physical requirements for good reconstructions are more stringent. At high dose levels, the data have sufficient SNR and the reconstruction quality becomes less sensitive to experimental parameters.

Ptychographic reconstructions of simulated bilayer $\mathrm{MoS}_{2}$ at different total electron doses

Figure 5. Ptychographic reconstructions of simulated bilayer MoS2\mathrm{MoS}_{2} at different total electron doses with an optimized set of experimental parameters compared with an expert-chosen set. (a-d) Phase maps of the reconstructed objects using experimental parameters that are optimized by Bayesian optimization. (e-h) Reconstructions using a fixed set of experimental parameters that are similar to ref.[13].

The results from BO allowed us to estimate the entire 4D parameter space and observe how optimal experimental conditions depend on the total electron dose. As shown in Figure 6, small probe and scan step size produce better results at extremely low-dose regimes. However, with increasing total electron counts, one can theoretically use a larger scan step size (>55 A˚\mathring{\mathrm{A}}) given sufficient probe overlap and detector pixels. Similarly, it is more advantageous to use a relatively small detector size (e.g. 128 ×\times 128) at a low dose as more pixels lead to poor SNR. Lastly, with the exception of 100 e−/A˚2e^{-}/\mathring{\mathrm{A}}^{2}, most of the optimal conditions found by BO have large (>30 mrad) aperture size, indicating that in addition to its size, the probe structure also influences the quality of ptychographic reconstructions. This agrees with previous literature[15] that shows specialized focusing optics can produce superior images. Because the focusing probe is typically characterized by a few physical parameters in electron microscopy, we believe the probe structure can be further optimized using the BO framework. This thus provides general guidance for future ptychography experiments.

Optimal experimental parameters for electron ptychography

Figure 6. Optimal experimental parameters for electron ptychography. At each electron dose level, BO attempts to optimize scan step size(a), detector size(b), probe defocus(c), and aperture size(d).

Discussion

As a general technique for black-box optimization, BO provides a framework that easily extends to other parameter tuning tasks beyond the eight reconstruction parameters and four experimental parameters studied in this work. For instance, in multi-slice ptychography, one can optimize model parameters such as the sample thickness and the number of layers. By minimizing the data error between reconstruction and data, BO facilitates automatic estimation of experimental conditions, including probe defocus or global scan position errors, which cannot be measured accurately by electron microscopes. In addition, our parameter tuning workflow has implications for similar inverse problems, including tomography or X-ray ptychography.

Currently, automatic parameter tuning and typical electron ptychography experiments operate on similar time scales (a few hours), which prohibits online parameter tuning. However, multiple hardware and software improvements can be made to further enhance the computational efficiency. First of all, because the majority of time in the workflow is used for ptychographic reconstruction, utilizing more advanced hardware, such as high memory-bandwidth GPUs can significantly reduce the processing time by more than tenfold compared to the current work. The total processing time can be further shortened by carrying out multiple reconstructions in parallel and using multi-points optimization strategies[39]. In addition, experienced scientists may leverage additional properties about reconstruction algorithms or data to reduce the parameter space in BO. The reduction can be implemented at the beginning or during the parameter tuning workflow. Lastly, recent developments such as physics-informed BO[40], causal BO[41], and deep kernel learning[24], may provide more solutions that facilitate more intelligent decision-making by exploiting underlying relationships between different parameters.

Lastly, we want to emphasize the importance of the objective function for optimizing reconstruction quality. An ideal objective function should reflect the accuracy of ptychographic reconstructions so that automatic parameter tuning produces true sample structures rather than artifacts. In numerical simulations, accuracy can be directly quantified since a ground truth is available. However, for experimental data, the complete information is unknown to researchers and many prevailing evaluation metrics (e.g. the FRC score) only characterize the precision of reconstructions. The FRC analysis is often used in ptychography literature[13, 25, 30, 32, 42] and correlates with accuracy to some extent, especially when the dominating factor is dose. Nevertheless, there exist reconstruction and experimental parameters that lead to deceiving results with high precision but low accuracy. For instance, applying image regularization techniques such as de-noising may “improve” the FRC score by removing noisy artifacts in the object, but reduce sharp features if the image is over-smoothed. To avoid such systematic bias, one should be attentive to limitations of different metrics and, if possible, incorporate additional knowledge into the parameter tuning workflow to directly optimize accuracy, or combine with precision measurement via multi-objective optimization[43]. Future works will explore other precision quantification methods that have better correlation with accuracy as well as simple computational complexity.

Conclusion

In summary, we demonstrated a human-out-of-loop parameter tuning framework for electron ptychography based on Bayesian optimization with Gaussian processes. The workflow does not require strong prior knowledge about the input data or advanced reconstruction techniques, and can automatically determine the parameters that correctly account for various experimental errors and produce high-resolution ptychographic reconstruction of experimental data. Similarly, BO can be used to search for the optimal experimental conditions in complex multi-dimensional parameter space, allowing better designs for ptychography applications such as low-dose imaging. With rapid developments in computing and control hardware, software, and advanced BO techniques, we anticipate that fully automatic parameter tuning will achieve sufficient throughput for real-time electron ptychography applications.

Acknowledgments

We thank Wendy Di for helpful discussions. Y.H. and M.C. acknowledge support from the Welch Foundation (C-2065-20210327). This research used resources of the Advanced Photon Source, a U.S. Department of Energy (DOE) Office of Science User Facility operated for the DOE Office of Science by Argonne National Laboratory under Contract No. DE-AC02-06CH11357.

Methods

Ptychographic reconstruction

Ptychographic reconstructions were carried out using a customized library based upon the PtychoShelves package[26]. The library, which is maintained at https://github.com/yijiang1/fold_slice, supports electron ptychography data and provides a python interface. For reconstruction parameter tuning studies, we further modified the code to allow for time-limited reconstruction instead of standard iteration-limited reconstruction.

Supplementary table 1 summarizes eight types of reconstruction parameters that are explored during automatic parameter tuning. These parameters influence both the quality and efficiency of ptychographic reconstruction and are frequently adjusted for different experimental data. The core algorithm is the maximum likelihood ptychography with a least-squares solver[25], which provides both Gaussian and Poisson probability distribution to model data noise. The method also used a mini-batch update strategy to efficiently balance reconstruction speed and convergence rate. Thus, the number of diffraction patterns in each batch and the batch selection scheme (sparse vs. compact) are tunable parameters in reconstruction. In addition, the number of probe modes in mixed-states ptychography[27] can be adjusted to account for partial coherence[13] and probe vibration[44]. In the orthogonal probe relaxation (OPR) technique[28], which is often used to reduce artifacts caused by probe variation in a single scan, the number of orthogonal modes kept in truncated singular value decomposition controls the amount of structural changes at each scan position. Moreover, position correction can refine inaccurate scan positions and intensity correction accounts for changes in probe intensity. Lastly, the ”multimodal” option specifies if all or only the first probe mode are used to update the object function.

In general, the upper bounds for the number of mixed-states probe modes, the OPR modes, and the batch size are limited by the data size and the GPU (NVIDIA GeForce GTX 1080 Ti) memory. For simplicity, we define position correction, intensity correction, and multimodal as binary variables. If an option is set to true, then feature is used throughout the entire reconstruction process.

Bayesian Optimization with Gaussian Process

Bayesian optimization was carried out with the Scikit Optimize library[45]. After each ptychographic reconstruction, the image quality and corresponding parameters are used to update the GP model. Here we used the Matern kernel[46] – a popular covariance function defined as:

k(xi,xj)=1Γ(ν)2ν−1(2νld(xi,xj))νKν(2νld(xi,xj))(1)k(x_i,x_j) = \frac{1}{\Gamma(\nu)2^{\nu-1}}\left(\frac{\sqrt{2\nu}}{l}d(x_i,x_j)\right)^\nu K_\nu\left(\frac{\sqrt{2\nu}}{l}d(x_i,x_j)\right) \tag*{(1)}

where d(xi,xj)d(x_i,x_j) measures the Euclidean distance between two points, Γ(ν)\Gamma(\nu) is the gamma function, and KνK_\nu is the modified Bessel function of the second kind. ν\nu is a positive parameter that controls the smoothness of the kernel and ll is the length scale, which is updated during BO. For all reconstructions parameter tuning studies in this paper, ν\nu is set to 1.

To sample the next point, we used a portfolio strategy known as “GP Hedge”[47], which selects points using a pool of acquisition functions, including negative probability of improvement[48], expected improvement[49], and upper confidence bound[50]. Lastly, the automatic parameter tuning workflow randomly sample a small number of initial points for Gaussian Process modeling before the Bayesian optimization process.

Experimental Parameter Tuning

For experimental parameter optimization, we generated a simulated potential of bilayer MoS2_2 with a 30∘30^\circ twist. Single-atom potentials[51] placed at appropriate coordinates were summed to generate the full potential of the bilayer. Interpolation was used to avoid the large singularity at the center of individual potentials. The resulting potential is 2048×20482048 \times2048 pixels with a pixel size of 0.125 Å.

For all simulated data, the scan field of view was about 66 nm ×6\times6 nm and the pixel size was fixed at 0.125 Å. The beam energy was set 80 keV. Each 4D dataset was simulated assuming the strong phase approximation, and then reconstructed with a single probe mode, compact batch selection scheme, and no additional corrections. The batch size was chosen dynamically to fully utilize the GPU memory. All reconstructions were run for 500 iterations on a single NVIDIA V100 GPU, and took from ∼10\sim10 seconds to ∼10\sim10 minutes, depending on the data size.

As summarized in supplementary Table 2, most experimental parameters are defined as continuous variables, giving an infinite number of possible parameter space. Bayesian optimization attempted to maximize the SSIM[38] between a reconstruction and the ground truth. The Hammersley sampling method[52] was used to explore 100 initial points that randomly cover the entire parameter, after which BO was used to search for additional 700 sets of parameters.

Figure1-a,b1-c,d1-e,f1-g,h
Reconstruction parameters
Number of mixed-state probe modes7771
Batch size30060300300
Batch selection schemeSparseSparseSparseSparse
Number of OPR modes0000
Intensity correctionFalseFalseFalseFalse
Position correctionTrueTrueFalseTrue
Noise modelGaussianGaussianGaussianGaussian
Modal updateFalseFalseFalseFalse
Reconstruction quality
Area under the FRC curve0.8180.7090.6010.239
1-bit FRC resolution (A˚\mathring{\mathrm{A}})0.2060.4330.4380.997
SSIM0.8960.8160.7910.518

Table 2. Reconstruction parameters and quality evaluations of bilayer MoSe2_2/WS2_2 sample. The parameters correspond to the reconstructions that are shown in Figure 2. The automatic parameter tuning with BO was used to optimize the area under the FRC curve, which reflects the similarity between two independent reconstructions. This is consistent with visual inspection and other metrics such as the 1-bit FRC resolution and SSIM.

Availability of Data and Materials

Diffraction data used for reconstruction parameter tuning are included in this published article by Chen et al.[13]. All other data and code are available from the corresponding author at reasonable request.

Supplementary Information

Michael C. Cao; Zhen Chen; Yi Jiang; Yimo Han

Figures

Eight scatter plots showing optimal reconstruction parameters as functions of time limit

Figure S1. Optimal reconstruction parameters of an experimental dataset of bilayer MoSe2/WS2\mathrm{MoSe}_2/\mathrm{WS}_2. At each time limit, eight different types of parameters were optimized by automatic parameter tuning with Bayesian optimization.

Tables

Experimental parameterRange
Scan step size (A˚\mathring{\mathrm{A}})1.5 to 15
Aperture size (mrad)5 to 35
Probe defocus (nm)0 to 1000
Detector size (# of pixels)64x64, 128x128, 196x196, ..., 1024x1024

Table 3. Experimental parameters and ranges in automatic parameter optimization for dose-limited electron ptychography.

References

  1. [1]Yang, H., Rutte, R., Jones, L., Simson, M., Sagawa, R., Ryll, H., Huth, M., Pennycook, T., Green, M., Soltau, H., et al.: Simultaneous atomic-resolution electron ptychography and z-contrast imaging of light and heavy elements in complex nanostructures. Nature Communications 7(1), 1–8 (2016)DOI
  2. [2]Gao, S., Wang, P., Zhang, F., Martinez, G.T., Nellist, P.D., Pan, X., Kirkland, A.I.: Electron ptychographic microscopy for three-dimensional imaging. Nature communications 8(1), 1–8 (2017)DOI
  3. [3]Jiang, Y., Chen, Z., Han, Y., Deb, P., Gao, H., Xie, S., Purohit, P., Tate, M.W., Park, J., Gruner, S.M., et al.: Electron ptychography of 2d materials to deep sub-ångström resolution. Nature 559(7714), 343–349 (2018)DOI
  4. [4]Chen, Z., Jiang, Y., Shao, Y.-T., Holtz, M.E., Odstrčil, M., Guizar-Sicairos, M., Hanke, I., Ganschow, S., Schlom, D.G., Muller, D.A.: Electron ptychography achieves atomic-resolution limits set by lattice vibrations. Science 372(6544), 826–831 (2021)DOI
  5. [5]Hoppe, W.: Beugung im inhomogenen primärstrahlwellenfeld. i. prinzip einer phasenmessung von elektronenbeugungsinterferenzen. Acta Crystallographica Section A: Crystal Physics, Diffraction, Theoretical and General Crystallography 25(4), 495–501 (1969)DOI
  6. [6]Ophus, C., Ercius, P., Sarahan, M., Czarnik, C., Ciston, J.: Recording and using 4d-stem datasets in materials science. Microscopy and Microanalysis 20(S3), 62–63 (2014)DOI
  7. [7]Tate, M.W., Purohit, P., Chamberlain, D., Nguyen, K.X., Hovden, R., Chang, C.S., Deb, P., Turgut, E., Heron, J.T., Schlom, D.G., et al.: High dynamic range pixel array detector for scanning transmission electron microscopy. Microscopy and Microanalysis 22(1), 237–249 (2016)DOI
  8. [8]Mir, J., Clough, R., MacInnes, R., Gough, C., Plackett, R., Shipsey, I., Sawada, H., MacLaren, I., Ballabriga, R., Maneuski, D., et al.: Characterisation of the medipix3 detector for 60 and 80 kev electrons. Ultramicroscopy 182, 44–53 (2017)DOI
  9. [9]Thibault, P., Dierolf, M., Menzel, A., Bunk, O., David, C., Pfeiffer, F.: High-resolution scanning x-ray diffraction microscopy. Science 321(5887), 379–382 (2008)DOI
  10. [10]Guizar-Sicairos, M., Fienup, J.R.: Phase retrieval with transverse translation diversity: a nonlinear optimization approach. Optics express 16(10), 7264–7278 (2008)
  11. [11]Maiden, A.M., Rodenburg, J.M.: An improved ptychographical phase retrieval algorithm for diffractive imaging. Ultramicroscopy 109(10), 1256–1262 (2009)DOI
  12. [12]Maiden, A.M., Humphry, M.J., Zhang, F., Rodenburg, J.M.: Superresolution imaging via ptychography. JOSA A 28(4), 604–612 (2011)DOI
  13. [13]Chen, Z., Odstrcil, M., Jiang, Y., Han, Y., Chiu, M.-H., Li, L.-J., Muller, D.A.: Mixed-state electron ptychography enables sub-angstrom resolution imaging with picometer precision at low dose. Nature communications 11(1), 1–10 (2020)DOI
  14. [14]Song, J., Allen, C.S., Gao, S., Huang, C., Sawada, H., Pan, X., Warner, J., Wang, P., Kirkland, A.I.: Atomic resolution defocused electron ptychography at low dose with a fast, direct electron detector. Scientific reports 9(1), 1–8 (2019)DOI
  15. [15]Pelz, P.M., Qiu, W.X., Bücker, R., Kassier, G., Miller, R.D.: Low-dose cryo electron ptychography via non-convex bayesian optimization. Scientific reports 7(1), 1–13 (2017)arxiv.org/abs/1702.05732
  16. [16]Zhou, L., Song, J., Kim, J.S., Pei, X., Huang, C., Boyce, M., Mendonça, L., Clare, D., Siebert, A., Allen, C.S., et al.: Low-dose phase retrieval of biological specimens using cryo-electron ptychography. Nature communications 11(1), 1–9 (2020)DOI
  17. [17]Chen, Z., Shao, Y.-T., Jiang, Y., Muller, D.: Three-dimensional imaging of single dopants inside crystals using multislice electron ptychography. Microscopy and Microanalysis 27(S1), 2146–2148 (2021)DOI
  18. [18]Rasmussen, C.E.: Gaussian processes in machine learning. In: Summer School on Machine Learning, pp. 63–71 (2003). SpringerDOI
  19. [19]Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyperparameter optimization. Advances in neural information processing systems 24 (2011)
  20. [20]Brochu, E., Cora, V.M., De Freitas, N.: A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599 (2010)arxiv.org/abs/1012.2599
  21. [21]Mahendran, N., Wang, Z., Hamze, F., De Freitas, N.: Adaptive mcmc with bayesian optimization. In: Artificial Intelligence and Statistics, pp. 751–760 (2012). PMLR
  22. [22]Duris, J., Kennedy, D., Hanuka, A., Shtalenkova, J., Edelen, A., Baxe- vanis, P., Egger, A., Cope, T., McIntire, M., Ermon, S., et al.: Bayesian optimization of a free-electron laser. Physical review letters 124(12), 124801 (2020)DOI
  23. [23]Zhang, C., Baraissov, Z., Duncan, C., Hanuka, A., Edelen, A., Maxson, J., Muller, D.: Aberration corrector tuning with machine-learning-based emittance measurements and bayesian optimization. Microscopy and Microanalysis 27(S1), 810–812 (2021)DOI
  24. [24]Roccapriore, K.M., Kalinin, S.V., Ziatdinov, M.: Physics discovery in nanoplasmonic systems via autonomous experiments in scanning trans- mission electron microscopy. arXiv preprint arXiv:2108.03290 (2021)DOI
  25. [25]Odstčil, M., Menzel, A., Guizar-Sicairos, M.: Iterative least-squares solver for generalized maximum-likelihood ptychography. Optics express 26(3), 3108–3123 (2018)
  26. [26]Wakonig, K., Stadler, H.-C., Odstrčil, M., Tsai, E.H., Diaz, A., Holler, M., Usov, I., Raabe, J., Menzel, A., Guizar-Sicairos, M.: PtychoShelves, a ver- satile high-level framework for high-performance analysis of ptychographic data. Journal of applied crystallography 53(2), 574–586 (2020)DOI
  27. [27]Thibault, P., Menzel, A.: Reconstructing state mixtures from diffraction measurements. Nature 494(7435), 68–71 (2013)DOI
  28. [28]Odstrcil, M., Baksh, P., Boden, S., Card, R., Chad, J., Frey, J., Brocklesby, W.: Ptychographic coherent diffractive imaging with orthogonal probe relaxation. Optics express 24(8), 8360–8369 (2016)DOI
  29. [29]Tsai, E.H., Usov, I., Diaz, A., Menzel, A., Guizar-Sicairos, M.: X-ray ptychography with extended depth of field. Optics express 24(25), 29089– 29108 (2016)DOI
  30. [30]Jiang, Y., Deng, J., Yao, Y., Klug, J.A., Mashrafi, S., Roehrig, C., Preiss- ner, C., Marin, F.S., Cai, Z., Lai, B., et al.: Achieving high spatial resolution in a large field-of-view using lensless x-ray imaging. Applied Physics Letters 119(12), 124101 (2021)DOI
  31. [31]Harauz, G., van Heel, M.: Exact filters for general geometry three dimensional reconstruction. Optik. 73(4), 146–156 (1986)
  32. [32]Vila-Comamala, J., Diaz, A., Guizar-Sicairos, M., Mantion, A., Kewish, C.M., Menzel, A., Bunk, O., David, C.: Characterization of high-resolution diffractive x-ray optics by ptychographic coherent diffractive imaging. Optics express 19(22), 21333–21344 (2011)
  33. [33]Van Heel, M., Schatz, M.: Fourier shell correlation threshold criteria. Journal of structural biology 151(3), 250–262 (2005)pubmed.ncbi.nlm.nih.gov/16125414
  34. [34]Egerton, R.: Control of radiation damage in the tem. Ultramicroscopy 127, 100–108 (2013)DOI
  35. [35]Russo, C., Egerton, R.: Damage in electron cryomicroscopy: Lessons from biology for materials science. MRS Bulletin 44(12), 935–941 (2019)DOI
  36. [36]Edo, T., Batey, D., Maiden, A., Rau, C., Wagner, U., Pešić, Z., Waigh, T., Rodenburg, J.: Sampling in x-ray ptychography. Physical Review A 87(5), 053850 (2013)DOI
  37. [37]Zhang, X., Chen, Z., Muller, D.: How many detector pixels do we need for super-resolution ptychography? Microscopy and Microanalysis 27(S1), 620–622 (2021)DOI
  38. [38]Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13(4), 600–612 (2004)pubmed.ncbi.nlm.nih.gov/15376593
  39. [39]Chevalier, C., Ginsbourger, D.: Fast computation of the multi-points expected improvement with applications in batch selection. In: International Conference on Learning and Intelligent Optimization, pp. 59–69 (2013). SpringerDOI
  40. [40]Ziatdinov, M., Ghosh, A., Kalinin, S.V.: Physics makes the difference: Bayesian optimization and active learning via augmented gaussian process. arXiv preprint arXiv:2108.10280 (2021)arxiv.org/abs/2108.10280
  41. [41]Aglietti, V., Lu, X., Paleyes, A., González, J.: Causal bayesian optimization. In: International Conference on Artificial Intelligence and Statistics, pp. 3155–3164 (2020). PMLRarxiv.org/abs/2005.11741
  42. [42]Holler, M., Diaz, A., Guizar-Sicairos, M., Karvinen, P., Färm, E., Härkönen, E., Ritala, M., Menzel, A., Raabe, J., Bunk, O.: X-ray ptychographic computed tomography at 16 nm isotropic 3d resolution. Scientific reports 4(1), 1–5 (2014)DOI
  43. [43]Jeong, S., Obayashi, S.: Efficient global optimization (ego) for multi-objective problem and data mining. In: 2005 IEEE Congress on Evolutionary Computation, vol. 3, pp. 2138–2145 (2005). IEEEDOI
  44. [44]Clark, J.N., Huang, X., Harder, R.J., Robinson, I.K.: Dynamic imaging using ptychography. Physical review letters 112(11), 113901 (2014)pubmed.ncbi.nlm.nih.gov/24702370
  45. [45]Head, T., MechCoder, G.L., Scherbatyi, I., et al.: scikit-optimize/scikit-optimize: v0. 5.2. Zenodo (2018)
  46. [46]Genton, M.G.: Classes of kernels for machine learning: a statistics perspective. Journal of machine learning research 2(Dec), 299–312 (2001)DOI
  47. [47]Hoffman, M., Brochu, E., de Freitas, N., et al.: Portfolio allocation for bayesian optimization. In: UAI, pp. 327–336 (2011). Citeseer
  48. [48]Kushner, H.J.: A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise (1964)
  49. [49]Mockus, J., Tiesis, V., Zilinskas, A.: The application of bayesian methods for seeking the extremum. Towards global optimization 2(117-129), 2 (1978)
  50. [50]Cox, D.D., John, S.: A statistical method for global optimization. In: [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics, pp. 1241–1246 (1992). IEEE
  51. [51]Kirkland, E.J.: Advanced Computing in Electron Microscopy vol. 12. Springer, New York (1998)
  52. [52]Hammersley, J.M.: Monte carlo methods for solving multivariable problems. Annals of the New York Academy of Sciences 86(3), 844–874 (1960)DOI

Paper details

vlnc:W_zug6777f
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents