WorksComputer Science
GRATEV2.0: Computational Tools for Real-time Analysis of High-throughput High-resolution TEM (HRTEM) Images of Conjugated Polymers
Abstract
Automated analysis of high-resolution transmission electron microscopy (HRTEM) images is increasingly essential for advancing research in organic electronics, where precise characterization of nanoscale crystal structures is crucial for optimizing material properties. This paper introduces an open-source computational framework called GRATEV2.0 (GRaph-based Analysis of TEM), designed for real-time analysis of HRTEM data, with a focus on characterizing complex microstructures in conjugated polymers, illustrated using Poly[N-9'-heptadecanyl-2,7-carbazole-alt-5,5-(4',7'-di-2-thienyl-2',1',3'-benzothiadiazole)] (PCDTBT), a key material in organic photovoltaics. GRATEV2.0 employs fast, automated image processing algorithms, enabling rapid extraction of structural features like d-spacing, orientation, and crystal shape metrics. Gaussian process optimization rapidly identifies the user-defined parameters in the approach, reducing the need for manual parameter tuning and thus enhancing reproducibility and usability. Additionally, GRATEV2.0 is compatible with high-performance computing (HPC) environments, allowing for efficient, large-scale data processing at near real-time speeds. A unique feature of GRATEV2.0 is a Wasserstein distance-based stopping criterion, which optimizes data collection by determining when further sampling no longer adds statistically significant information. This capability optimizes the amount of time the TEM facility is used while ensuring data adequacy for in-depth analysis. Open-source and tested on a substantial PCDTBT dataset, this tool offers a powerful, robust, and accessible solution for high-throughput material characterization in organic electronics.
Introduction
Microscopy has long been a cornerstone in materials science, offering a unique window into the microstructure and enabling scientists to study properties at various scales, from the millimeter down to the atomic level [1, 2]. By visualizing otherwise inaccessible structures, microscopy provides critical insights into how atomic and molecular arrangements influence key macroscopic properties such as mechanical strength, electrical conductivity, and chemical reactivity [3]. This fundamental understanding is essential for designing advanced materials with tailored properties for applications in fields such as electronics, energy storage, and catalysis.
High-resolution transmission electron microscopy (HRTEM) has advanced significantly in recent years, transforming nanoscale imaging and allowing researchers to capture atomic-level details of materials [4]. HRTEM can now achieve sub-angstrom spatial resolution, making it possible to directly observe atomic lattices, defects, and interfaces that govern a material’s behavior [5]. The development of automated data acquisition systems has further broadened HRTEM’s capabilities, enabling the collection of extensive datasets comprising hundreds or even thousands of high-resolution images [6]. These technological advances have opened new avenues for studying complex materials, such as organic semiconductors and conjugated polymers, with applications in organic electronics and photovoltaics [7].
While modern HRTEM has enhanced our ability to study complex materials, it also introduces challenges in data management and analysis due to the sheer volume and complexity of the data generated [8]. The scale of high-resolution datasets can quickly overwhelm traditional analysis workflows, which are often manual and time-consuming. This manual approach is highly dependent on the expertise and subjective judgment of the experimentalist, making it challenging to ensure consistency and reproducibility. In high-throughput applications where rapid feedback is essential, such as optimizing synthesis conditions or tracking real-time structural changes, the limitations of manual analysis become particularly pronounced.
To address these challenges, automated methods for HRTEM data analysis have emerged over the past decade. These methods aim to extract quantitative structural information from digital micrographs with minimal human intervention, improving both the efficiency and reliability of analysis [3, 9, 10, 11, 12]. Automated approaches can be broadly categorized into offline (post-acquisition) and online (real-time) methods. Offline methods involve processing data after acquisition, which is useful for extracting detailed structural information but may be computationally intensive and unable to keep pace with data acquisition rates [9, 11, 12, 13]. A motivating example is our prior work – GRATE (GRaph based Analysis of TEM images) [13] – that is based on Matlab. The computational and memory overhead makes offline methods less suitable for high-throughput applications where rapid processing is needed. The reliance on post-acquisition processing has underscored the need for integrating HRTEM with high-performance computing (HPC) resources to handle large volumes of data efficiently.
In recent years, there has been a strong push toward developing in situ, or real-time, automated analysis methods [14, 15, 16]. In situ analysis allows for data interpretation during experiments, providing immediate feedback that can guide adjustments in experimental parameters. This capability is particularly valuable in dynamic experiments, such as observing structural changes in response to external stimuli or monitoring materials during synthesis [17, 18, 19, 20, 21, 22, 23]. Real-time analysis requires methods that can handle high-resolution data quickly and accurately, maintaining performance under the demanding conditions of live data acquisition.
Machine learning (ML) has shown promise in automating feature recognition in HRTEM images, with deep learning models capable of identifying patterns and structures in high-resolution data [24, 25, 26]. However, ML approaches have limitations in this context. They typically require large, annotated datasets for training, which is resource-intensive and time-consuming to create [25, 27, 28]. Each chemical system may exhibit unique structural features and imaging artifacts, requiring retraining or fine-tuning of models to ensure accuracy [9, 12]. This dependence on extensive training data reduces the adaptability of ML methods across different material systems, particularly in high-throughput or exploratory research where materials and imaging conditions vary frequently. Recent work in creating foundational models, and machine learning approaches like transfer learning and few-shot learning applied to these problems show promise.
In contrast, robust image processing techniques provide a more flexible and efficient solution for automated analysis. Image processing-based approaches can generate interpretable, reproducible data in real time without the need for extensive, system-specific training. These methods typically involve well-established procedures like frequency filtering, binarization, and morphological operations that are adaptable across diverse materials [9, 11, 13, 29, 30, 31]. In conjunction with ML models, image processing methods offer a practical solution for rapid, in situ data analysis with minimal human input, making them particularly valuable for high-throughput studies in materials science.
GRATEv2 aims to bridge the gap between offline and real-time analysis in high-resolution transmission electron microscopy (HRTEM) by providing an automated, image-processing-based framework with minimal human intervention. Tailored for high-throughput settings, GRATEv2 combines rapid data extraction with a robust and user-friendly parameter optimization process. By focusing on image processing techniques augmented with Gaussian process optimization, GRATEv2 minimizes the need for manual parameter selection and tuning, enhancing reproducibility and accessibility for researchers.
A schematic overview of the GRATEv2 computational framework is shown in Figure 1a. The framework processes raw HRTEM images of PCDTBT (Figure 2a), applies preprocessing, and performs automated image processing with parameters optimized via Bayesian optimization (Figure 1b). Traditionally, manual annotation of crystals in HRTEM images is time-consuming and subjective (Figure 2). In our approach, only a dozen manually annotated images (Figure 2b) are used as input to a Bayesian optimizer to rapidly identify material-specific image processing parameters. This minimal requirement significantly reduces the burden on researchers, facilitating rapid deployment of the algorithm on new datasets.
Figure 1. Schematic overview of the GRATEv2 computational framework. The framework processes raw HRTEM images of PCDTBT, applies preprocessing, and performs automated image processing with parameters optimized via Bayesian optimization. A data sufficiency criterion based on the Wasserstein distance assesses whether additional TEM data is needed. The output comprises extracted structural features such as d-spacing, orientation, and crystal shape metrics. (a) The overall computational framework of GRATEv2, and (b) the detailed Bayesian Optimization (BO) component used for parameter tuning.

Figure 2. Comparison between (a) the original HRTEM image and (b) the manually annotated crystals used for training the Bayesian model. 13 manually annotated images are used for effective training in the work. Manual annotation is time-consuming and subjective.
GRATEv2 introduces several key innovations to address the limitations of current HRTEM analysis methods:
GRATEv2 offers fast and efficient processing of HRTEM images on the order of a few seconds per image. By supporting batch processing and multiprocessing, it enables rapid analysis of large datasets, significantly improving throughput.
It extends advanced image analysis techniques to organic polymer materials, specifically PCDTBT, which have not been extensively studied with these methods.
Bayesian optimization is employed within GRATEv2 to automate the tuning of material-specific image processing parameters, allowing the framework to adapt to different datasets with minimal expert input (Figure 1b).
The algorithm parameters are constructed as functions of known -spacing values, simplifying parameter selection and making the method more accessible to users without extensive image processing expertise. This also ensures that parameter selection is interpretable and, thus, scientifically justified.
GRATEv2 incorporates a data sufficiency criterion based on the Wasserstein distance to guide data collection efforts. This criterion provides a quantitative stopping point, indicating when further TEM data collection no longer yields additional insights, which is important when access to imaging facilities is limited and imaging is expensive.
By optimizing data collection, GRATEv2 helps experimentalists avoid unnecessary resource expenditure while ensuring data quality. This combination of fast, automated processing, real-time adaptability, and efficient data collection positions GRATEv2 as a powerful tool for materials research, particularly in the study of organic electronic materials such as conjugated polymers [32, 33, 34].
Results
We first provide brief details of the material and process used to collect the HRTEM data for completeness. To prepare TEM samples, 5 mg/mL solutions of PCDTBT were dissolved in chlorobenzene within a nitrogen glovebox at 45°C for at least 12 hours. Silicon wafers were cleaned by sonication for 20 minutes in acetone, followed by 20 minutes in isopropanol, and subsequently subjected to UV-ozonation for 20 minutes. Poly(3,4-ethylenedioxythiophene)-poly(styrene sulfonate) (PEDOT:PSS) films were cast onto silicon substrates by spin coating at 4000 RPM for 2 minutes in air to serve as a sacrificial layer, facilitating film floating. Substrates were then transferred to a nitrogen glovebox, where PCDTBT films were spin-cast at 800 RPM for 2 minutes. For TEM sample preparation, the coated substrates were removed from the glovebox, floated in deionized water, and carefully transferred onto copper TEM grids. These samples were left under ambient conditions to dry overnight and then annealed in the nitrogen glovebox at 190 °C for 2 hours. High-resolution TEM (HRTEM) imaging was performed at the Penn State Materials Characterization Lab using the FEI Titan Krios microscope operating at 300 kV, equipped with the K2 direct electron detector and a cryo-stage. A dose rate of was used for a 2.5 s exposure. An automated acquisition process was set to autofocus before each capture at 300 kx magnification, with a randomly assigned defocus value between 0 and . Images were acquired at 470 kx magnification with a step size between exposed regions, resulting in a total of 637 images for subsequent analysis.
Using GRATEv2, we detected crystals within each of the HRTEM images. For each identified crystal, the following features were extracted (see § 2.2): center-of-mass coordinates, orientation angle relative to the image axis, d-spacing, and crystal lengths along both the major and minor axes. Through this process, GRATEv2 identified a total of 4350 ordered domains from the HRTEM images. In § 2.3, we present the timing performance of GRATEv2, along with the time distribution among different components of the analysis process. In § 2.4, we evaluate the data sufficiency—that is, how many images are sufficient to achieve statistical convergence in the extracted features.
Advantages of Bayesian Parameter Optimization
To evaluate the effectiveness of integrating Bayesian optimization for hyperparameter tuning in our image processing algorithm, we conducted a comparative analysis between manually selected parameters and the Bayesian-optimized parameters. The performance was assessed using the Intersection over Union (IoU) metric between the algorithm’s output against expert-annotated ground truth on six representative HRTEM images.
Quantitative Comparison of IoU Scores Table 1 presents the IoU scores for each image using both the manually selected parameters and the Bayesian-optimized parameters. Additionally, the table includes the differences in IoU scores between the two parameter sets for each image. The average and standard deviation of IoU score across the six images is also calculated for both cases.
| Image Filename | Manual Selected Parameters IoU | Bayesian Parameters IoU | IoU Difference, |
| 1.tif | 0.5296 | 0.6911 | +0.1615 |
| 2.tif | 0.3274 | 0.5156 | +0.1882 |
| 3.tif | 0.3178 | 0.4934 | +0.1756 |
| 4.tif | 0.5285 | 0.5864 | +0.0579 |
| 5.tif | 0.3906 | 0.5400 | +0.1494 |
| 6.tif | 0.5288 | 0.6438 | +0.1150 |
| Average IoU | 0.4371 | 0.5784 | +0.1413 |
Table 1. IoU Scores over validation dataset for Manual and Bayesian-Optimized Parameters with Differences
The average IoU score using the Bayesian-optimized parameters is 0.5784, representing an improvement of approximately 32.3% over the average IoU of 0.4371 obtained with the manually selected parameters. This significant increase demonstrates the effectiveness of Bayesian optimization in enhancing the algorithm’s performance in detecting and segmenting crystalline regions in HRTEM images. The computed -value corresponding to -statistic is 7.074, which exceeds the critical -value of 2.571, indicating strong statistical justification for improvement.
Visual Comparison of Detection Results Figure 3 and Appendix § 12 provide qualitative comparisons between the segmentation results obtained with manually selected parameters and those obtained with Bayesian-optimized parameters. Visual comparisons illustrate that the algorithm with Bayesian-optimized parameters generally provides a more accurate and comprehensive detection of crystalline regions than the manually tuned approach. The Bayesian-optimized parameters tend to align the detected crystal boundaries more consistently with the annotated features. In particular, the algorithm with Bayesian-optimized parameters appears to capture finer details and produce clearer delineation of crystal boundaries, suggesting a closer correspondence to the expert annotations than that achieved under manual parameter tuning.
Figure 3. Figure 3: Comparison of Ground Truth, Manually selected parameter detection, and Bayesian-Optimized parameter detection across three different images. Each column represents a distinct detection method, illustrating how Bayesian optimization enhances segmentation accuracy by more closely matching the ground truth annotations compared to manual parameter tuning. The Bayesian optimization process was conducted over 200 iterations, achieving a minimum loss value of -0.7319 at the 151st evaluation.
Convergence of the Bayesian Optimization Process Figure 4 depicts the convergence of the Bayesian optimization process. The objective function, defined as the negative mean IoU, decreases over successive iterations, indicating that the optimization algorithm effectively identifies hyperparameters that enhance the segmentation performance. The convergence plot demonstrates that significant improvements are achieved within the first 51 iterations, after which the objective function gradually approaches a plateau.
Figure 4. Convergence of the Bayesian optimization process over 200 Iterations, illustrating the reduction in the loss function (negative IoU) as optimization progresses. The y-axis represents the minimum loss value achieved up to that evaluation. The minimum loss value of -0.7319 was attained at the 151st evaluation.
Analysis of Optimized Parameters Appendix § 10 compares the values of the manually selected parameters with those obtained through Bayesian optimization.
Several notable differences are observed between the parameter sets:
Morphological Operations: The Bayesian-optimized parameters for
closing_k_sizeandopening_k_sizeare significantly smaller than the manual values (2 vs. 15 and 17, respectively). This suggests that less aggressive morphological operations preserve finer details in the images, contributing to improved segmentation accuracy.Edge Detection and Filtering: Parameters like
ellipse_len_propConsanddspace_bandpassare adjusted to better capture the characteristics of the crystalline structures. The increase inellipse_len_propConsfrom 1.5 to 4.03 indicates a preference for detecting longer ellipses, aligning with the elongated shapes of crystals.Thresholding Parameters: The
pixThresh_propConsis slightly higher in the Bayesian-optimized parameters, which may help in differentiating crystals from the background noise more effectively.
Advantages of Bayesian Optimization The integration of Bayesian optimization into our image processing framework offers several advantages:
Enhanced Performance: The Bayesian-optimized parameters significantly improve the mean IoU score by approximately 32.3%, indicating superior segmentation performance.
Automated Tuning: Bayesian optimization automates the hyperparameter tuning process, reducing the need for manual intervention and deep image-processing expertise, thereby saving time and resources. This is especially important when the number of hyperparameters (here, 13) makes manual exploration rather tedious.
Efficient Exploration: The optimization process efficiently explores the hyperparameter space, converging to optimal values in fewer evaluations compared to exhaustive search methods. This is evident from the convergence plot in Figure 4.
Robustness: The optimized parameters produce robust results across various images, as evidenced by consistent improvements in IoU scores.
Structural Feature Extraction of Crystals from HRTEM Images
Using the parameter set listed in Appendix § 10, we applied GRATEv2 to detect and analyze crystalline domains within the HRTEM images of PCDTBT. The segmentation results for two images are presented in Figure 5, where the identified crystals are highlighted. Each detected crystal is surrounded by a convex hull boundary, with a shaded region representing a more precise delineation of the crystallite. A line at the crystal centroid indicates its orientation. The extracted properties for these detected crystals are summarized in Table 2, including their centroid coordinates, area, orientation angle, d-spacing, major and minor axis lengths, and axis angle. Additionally, the correlation measurements between pairs of crystals are provided in Table 3, featuring metrics such as metric distance, direct distance, and relative angle.
| Name | Centroid () | Area () | Angle () | d-Spacing () | MajorAxis () | MinorAxis () | AxisAngle () |
| 1.tif | (1748, 670) | 589.7 | -164.7 | 2.1 | 21.1 | 10.2 | 23.8 |
| (785, 1992) | 293.9 | -55.9 | 2.0 | 14.8 | 9.9 | -65.5 | |
| 2.tif | (534, 497) | 177.9 | -137.9 | 1.9 | 12.5 | 5.3 | 53.0 |
| (1607, 402) | 84.3 | -136.0 | 0.8 | 7.9 | 4.3 | 35.4 | |
| (3345, 546) | 71.7 | -148.8 | 1.7 | 14.4 | 4.8 | 39.6 | |
| (2050, 1975) | 125.4 | -146.0 | 2.2 | 16.0 | 4.2 | 34.1 | |
| (1922, 3396) | 263.8 | -141.4 | 2.0 | 14.2 | 8.0 | 46.9 |
Table 2. Features of the detected crystals shown in Figure 5.
| Name | Metric Distance (1) | Direct Distance () | Relative Angle () |
| 1.tif | 0.89 | 20.84 | 71.24 |
| 2.tif | 2.91 | 35.81 | 10.95 |
| 1.95 | 26.97 | 8.13 | |
| 2.45 | 40.94 | 3.5 | |
| 2.21 | 24.57 | 2.81 | |
| 2.91 | 40.58 | 7.45 | |
| 1.17 | 18.18 | 4.64 |
Table 3. Crystal correlation measurements for Figure 5.
Figure 5. (a) and (b) are 1.tif and 2.tif images respectively corresponding to Table 2 and Table 3. (a) and (b) shows the original image (left) and the segmentation output (right) from our algorithm for HRTEM of PCDTBT. The detected crystals have a d-spacing of 1.9nm. The image on the left is the input to the algorithm, and on the right is the output of the algorithm. Each detected crystal in the output shows (1) the convex hull boundary around the crystal, (2) the shaded region representing a more exact crystal region, and (3) a straight line at the centroid of the convex hull, which shows the orientation of the crystal patterns.
The detection of these crystalline domains allows for a comprehensive analysis of the microstructural features of PCDTBT. The d-spacing values shown in Figure 6b range from approximately 1.1nm to 2.9nm, which are consistent with the expected lattice spacings associated with PCDTBT’s crystalline structures [35]. Variations in d-spacing among the detected crystals may be attributed to differences in crystallite orientation, strain within the material, or inherent structural disorder due to the semi-crystalline nature of PCDTBT.
Figure 6. Individual crystal analysis. The properties of the individually detected crystals are plotted: (a) Crystal area, (b) FFT evaluated -spacing, (c) Angle difference between crystal’s pattern and its major axis, (d) Crystal aspect ratio, and (e) crystal axis lengths. (a) to (d) are histograms with kernel density estimate, and (e) is a scatter plot from analysis of the entire dataset.
The areas of the detected crystals vary significantly as shown in Figure 6a, with values ranging from approximately to . Larger crystal areas may correlate with improved charge transport properties, as larger crystalline domains can facilitate more efficient charge carrier mobility along the polymer chains [32]. The aspect ratios, derived from the major and minor axis lengths, provide insights into the shapes of the crystals. Higher aspect ratios indicate elongated, rod-like crystals, while lower aspect ratios suggest more equiaxed or spherical shapes. The diversity in crystal shapes and sizes can impact the overall morphology and performance of the polymer in electronic applications.
Figure 6 also presents additional statistical analyses of individual crystal properties across the dataset, including histograms of crystal areas, -spacings, orientation angles, and shape descriptors like aspect ratio. These plots allow us to identify prevalent features and distributions within the material. For example, the histogram of -spacings may show a peak around a specific value, indicating a dominant crystalline phase or preferred stacking distance within the polymer chains.
The comprehensive analysis provided by GRATEv2 enables us to establish quantitative relationships between microstructural features and potential material performance. By systematically characterizing the size, shape, orientation, and spatial distribution of crystalline domains, we can correlate these features with electronic properties measured in devices. This level of detailed microstructural understanding is essential for guiding the design and processing of conjugated polymers to achieve optimal performance in organic electronic applications.
For a more extensive and detailed exploration of this PCDTBT dataset, including insights into intercrystalline correlations and preferred crystallographic alignments, readers are referred to the work of our collaborators in [36]. Their study leverages automated HRTEM and the GRATEv2 image processing outputs to unravel how neighboring crystals preferentially align along certain lattice directions, likely reflecting underlying liquid crystalline order within the polymer. By combining the analysis presented here with their comprehensive assessment of orientation correlations and lattice parameters, one obtains a richer and more complete understanding of the polymer’s nanoscale structure and its implications for organic electronics.
Timing Statistics
The algorithm was executed on a computer equipped with a 96-core AMD EPYC 9654 CPU @ 3.7 GHz running Linux OS. The total time for processing a single high-resolution transmission electron microscopy (HRTEM) image with 1.9 nm -spacing crystals is approximately 6.52 seconds when utilizing a single core. The timing consumption by various parts of the algorithm is presented in Figure 7. The most time-consuming steps are skeletonization (approximately 4.66 seconds), followed by breaking branches and the preprocessing steps (blurring, histogram equalization, and thresholding). Utilizing all 96 cores, we processed an entire dataset of 637 images in 284 seconds (approximately 4 minutes and 44 seconds), reducing the per-image processing time to just 0.44 seconds. This significant improvement demonstrates the scalability and efficiency of our algorithm when parallelized across multiple cores.
Figure 7. Time taken by each step in the algorithm for the analysis of 1.9 nm -spacing crystals in a single image using a single core. The total processing time is approximately 6.52 seconds.
The performance enhancements of our algorithm are attributed to the use of optimized libraries and functions that efficiently handle computationally intensive tasks. Specifically, we employed:
The
skeletonizefunction fromskimage.morphology[37] for efficient skeletonization.OpenCV [38] functions
equalizeHistandthreshold(cv2.equalizeHist,cv2.threshold) for fast histogram equalization and image thresholding.Morphological operations such as closing and opening using
cv2.morphologyExfrom OpenCV.The
skeleton_to_csgraphfunction from theskan[39] library for converting skeleton images to graph representations.The
labelfunction fromskimage.measure[37] for rapid image segmentation.
Sparse graph structures and connected component analysis using
csr_matrixandconnected_componentsfromscipy.sparse.csgraph[40] for efficient graph construction and evaluation.
Fast Fourier transforms and other numerical operations using NumPy functions such as
np.fft[41].
The
alphashapelibrary to create shrink-wraps around point clouds.
By utilizing these optimized libraries, we minimized computational overhead and maximized the efficiency of each processing step. The skeletonization step, although still the most time-consuming, benefits significantly from the optimized implementation in skimage [37]. Similarly, the use of sparse matrices and efficient graph algorithms from scipy.sparse.csgraph [40] greatly accelerates the analysis of the skeleton’s connectivity.
Our analysis of the timing statistics reveals that the skeletonization step accounts for approximately 71% of the total processing time for a single image when executed on a single core. This indicates that skeletonization is a major computational bottleneck in the algorithm. However, due to the parallel nature of image processing tasks, distributing the workload across multiple cores significantly mitigates this bottleneck. By processing images concurrently, we effectively utilize the available computational resources, resulting in a substantial reduction in total processing time.
Furthermore, the efficient handling of large data structures, such as sparse matrices in graph construction and connected component analysis, contributes to the algorithm’s scalability. The use of optimized libraries ensures that even computationally intensive tasks are executed as efficiently as possible. This optimization is crucial when dealing with large datasets, as it enables rapid analysis without compromising accuracy or resolution.
The combination of algorithmic efficiency, optimized library functions, and effective parallelization allows our method to achieve high performance in processing and analyzing large volumes of HRTEM images. This capability is essential for applications requiring rapid data analysis and real-time feedback in materials science and related fields.
Data Sufficiency Analysis of PCDTBT Crystals in HRTEM Images
In our analysis of data sufficiency, we examine how the distribution of crystal areas reaches an asymptotic distribution as we incorporate more HRTEM images of PCDTBT. The complete dataset consists of 837 crystals from around 600 images. To evaluate how quickly the underlying distribution of crystal areas converges, we incrementally add data in fixed-size batches and compute the first-order Wasserstein distance between distributions formed by consecutive increments.
Specifically, we consider four different batch sizes at which images (rather, crystals) are added to the dataset: 10, 21, 42, and 84 crystals. For each batch size scenario, we start from 0 crystals and progressively add data in increments equal to the batch size until we reach the full dataset size (of 837 crystals). After each increment, we compute the Wasserstein distance between the new distribution (including crystals) and the previous distribution (with crystals). To obtain a more stable and reliable statistic, we repeat this evaluation 10 times at each increment, each time randomly sampling the batches from batches of crystals and then average the resulting distances.
As shown in Figure 8, the histograms become smoother and more stable as the dataset grows, demonstrating that the distribution of crystal areas converges toward a steady form. Meanwhile, Figure 8e shows how the averaged Wasserstein distance between consecutive increments decreases with additional data. Each curve represents a different batch size, revealing that the scale of incremental changes to the distribution depends on how many crystals are added at once.
Figure 8. Data sufficiency evaluation for different crystal areas.
Larger batch sizes (e.g., 84 crystals) introduce more data at each increment, leading to more pronounced changes in the distribution per step. Once enough large increments have been included, the distribution may show a sudden, relatively large drop in Wasserstein distance, then rapidly converge. In contrast, smaller batch sizes (e.g., 10 crystals) add data more gradually, producing smoother and more frequent updates. Each small increment makes a subtler change to the distribution, resulting in a more gradual and fine-grained trajectory toward convergence.
Because each batch size scenario scales the increments differently, the threshold for deciding when to stop data collection should also be scaled accordingly. For larger batch sizes, even a modest Wasserstein distance value (e.g., around 4 units for the 84-crystal batch) might indicate sufficient convergence, since one large increment can naturally shift the distribution more. For smaller batch sizes, where each increment is gentler (e.g., around 1.5 units difference for the 10-crystal batch), a lower threshold might be more appropriate, reflecting the finer control and resolution over the distribution’s shape. To illustrate this, Table 4 shows the Wasserstein distance between the full dataset and a dataset that is one batch smaller, for each considered batch size scenario. These values provide concrete examples of how batch size influences the scale of change in the distribution.
| Batch Size | Wasserstein Distance (Full vs. One Batch Less) |
| 84 | 4.12 |
| 42 | 2.83 |
| 21 | 2.08 |
| 10 | 1.51 |
Table 4. Wasserstein distance between the full dataset distribution and the distribution obtained by removing one batch of data from the full set, for various batch sizes.
With this information, an experimentalist might set a higher threshold for larger batch sizes (e.g., around 4 units for an 84-crystal batch) and a lower threshold for smaller batch sizes (e.g., closer to 1.5 units for a 10-crystal batch). This ensures that the threshold for deciding when to cease data collection remains proportionate to the scale of changes induced by each incremental addition of data.
In practical terms, experimentalists can use insights from Figure 8e and Table 4 to tailor their data collection strategy. By selecting an appropriate batch size and a corresponding Wasserstein distance threshold, they can decide when further data provides diminishing returns. If quick feedback and high resolution of distribution changes are desired, a smaller batch size and a lower threshold can be chosen. If time or resources are limited, larger batch sizes and a slightly higher threshold may be more suitable. In either case, once the averaged Wasserstein distance consistently falls below the chosen threshold, the experimentalist can confidently cease data collection, knowing the distribution is sufficiently representative. This approach transforms data sufficiency from a guesswork exercise into a clear, data-driven criterion that guides experimental resource allocation and ensures that the resulting dataset meets the necessary statistical rigor. An experimentalist might:
Begin Data Collection in Batches: They determine a batch size (e.g., 10 crystals per batch) and start collecting HRTEM images in increments of that batch size.
Periodic Assessment: After each new batch of data, they compute the Wasserstein distance between the current and previous distributions of crystal areas. This computation can be done after every increment, providing immediate feedback on the impact of newly acquired data.
Decision Point: If, after a certain number of increments, the averaged Wasserstein distance consistently falls below the established threshold (e.g., 1.5 units), the experimentalist has quantitative evidence that adding more data is unlikely to yield new insights into the crystal area distribution.
Stopping Data Collection: With this statistical criterion, the experimentalist can confidently stop collecting further HRTEM images, reallocating their time and resources. This prevents unnecessary prolonged imaging campaigns.
Conclusions
In this work, we have developed and presented GRATEv2, an open-source computational framework for the automated analysis of high-resolution transmission electron microscopy (HRTEM) images, specifically focusing on complex microstructures in conjugated polymers like PCDTBT. By leveraging fast, automated image processing algorithms augmented with Gaussian process optimization, GRATEv2 significantly reduces the need for manual selection of parameters and tuning, enhancing both reproducibility and user accessibility. The integration of a Wasserstein distance-based stopping criterion within GRATEv2 provides a quantitative method for optimizing data collection, ensuring efficient use of transmission electron microscopy (TEM) resources without compromising data quality.
GRATEv2’s compatibility with HPC environments allows for efficient, large-scale data processing at near real-time speeds, making it suitable for high-throughput applications in materials science. By successfully applying GRATEv2 to a substantial PCDTBT dataset, we demonstrated its efficacy in rapidly extracting critical structural features such as d-spacing, orientation, and shape metrics. This capability is particularly valuable for advancing research in organic electronics, where precise nanoscale characterization is essential for optimizing material properties.
Overall, GRATEv2 addresses key limitations of existing HRTEM analysis methods by providing a fast, adaptable, and user-friendly tool that enhances the efficiency and reliability of microstructural characterization. By making GRATEv2 open-source, we aim to facilitate its adoption and further development by the research community. Future work could involve extending GRATEv2 to other material systems and incorporating additional analytical capabilities, thereby broadening its applicability and impact in the field of materials characterization.
Methods
HRTEM Sample Preparation and Measurement Method
PCDTBT was synthesized using previously published procedures for Suzuki polycondensation in a Schlenk reactor flask[42, 43]. Sigma-Aldrich supplied all reactants. Polymerization occurred between 9-(9-heptadecanyl)-9H-carbazole-2,7-diboronic acid bis(pinacol) ester and 4,7-bis(2-bromo-5-thienyl)-2,1,3-benzothiadiazole in toluene, with equimolar amounts of each monomer. All other synthesis and purification procedures remained unaltered compared to the cited sources. The synthesis product was characterized by H nuclear magnetic resonance at 500 MHz[44].
5 mg/mL solutions of PCDTBT and chlorobenzene (Sigma-Aldrich) were prepared in a nitrogen glovebox and mixed overnight on a hotplate at 45°C. Silicon wafers were sonicated in acetone for 20 minutes, then isopropanol for 20 minutes. The wafers then underwent 20 minutes of ultraviolet light ozonation. A PEDOT:PSS (Clevios P and H.C. Starck) and water solution were then spin-coated onto the clean substrates at 4000 RPM for 2 minutes. The samples were then brought into the glovebox, and the heated PCDTBT solution was spin-coated on top of the PEDOT:PSS layer at 800 RPM for 2 minutes. The sample was then cut into squares and floated off in the water, and samples were collected on copper TEM grids. Samples were left to dry overnight and then annealed inside a nitrogen glovebox.
High-resolution imaging experiments were conducted on the Titan Krios microscope at the Penn State Materials Characterization Laboratory. The accelerating voltage was 300 kV, and the detector was a Falcon 3EC direct electron detector in counted mode. Regions of interest were spaced 2.5 µm apart and visually inspected on the atlas image for tears and defects before acquisition. The spot size was set to 5, and autofocus was done at 300 kx magnification before being increased to 470 kx for acquiring a 2.5 second exposure. The microscope produced a 650 nm beam with a dose rate of 50 e/Ųs.
Quantities of Interest
The inputs to the algorithm are 1) the HRTEM image 2) Approx d-spacings of the crystals to detect 3) The resolution of the image 4) Process parameters. The algorithm outputs the segmentation result for each input and a CSV file containing the feature details as listed in Table 2 and Table 3.
The distance between the crystals are defined as below:
| Symbol | Description |
| Metric distance: A dimensionless measure of the relative separation between two crystals, calculated by dividing the center-to-center distance by the sum of the radii of the crystals. | |
| Direct distance (nm): The actual center-to-center distance between two crystals, measured in nanometers (nm). | |
| Center-to-center crystal distance (nm): The straight-line distance between the centers of two crystals, measured in nanometers. | |
| Radii of the crystals (nm): The radii of each crystal in nanometers, where represents the radius of the first crystal and represents the radius of the second. |
Table 7 (PDF p. 14).
Image Processing Algorithm
Figure 9 illustrates the flowchart of GRATEv2 algorithm along with the associated process parameters. The process parameters are indicated in blue boxes, while the intermediate steps corresponding to Figure 10 are marked with brown numeric annotations.

Figure 9. Algorithm flowchart with parameters. Blue boxes are the process parameters and brown boxes are the intermediate steps shown in Figure 10.
Figure 10. Visualization of intermediate and final algorithm outputs.
The algorithm parameters are classified into two categories: primary parameters and secondary parameters. The primary parameters are independent values provided by the user based on the dataset’s imaging conditions and material properties, specifically the crystal d-spacing value and the image resolution. These parameters scale the key process parameters, enabling the algorithm to generalize across different imaging conditions and crystals of interest. They may vary significantly between datasets. Secondary parameters correspond to individual steps of the algorithm and are set to optimal values; they may require fine-tuning when changing datasets.
The algorithm is designed with several key objectives in mind: to generate a clear and concise representation of the essential information in the image; to enhance data quality by filtering out noise and irrelevant details; to convert the processed image data into a graph structure for advanced analysis; to use graph algorithms to identify clusters of polymer backbones forming crystallites; and to apply Fourier transform techniques to determine the d-spacing values.
Each step of the algorithm is detailed below, along with its associated parameters.
Initially, blurring is applied to smooth the image and reduce sharpness, which helps in minimizing noise and preparing the image for subsequent processing. This operation is performed multiple times, with the number of blurring iterations specified as a parameter. Following blurring, histogram equalization is employed to increase the image contrast, making it sharper and enhancing the distinction between polymer chains and the background. Improved contrast facilitates more accurate thresholding in the subsequent step.
The next step involves image thresholding using Otsu’s method to create a binary representation of the image, where polymer chains appear as black regions and the background as white. The result of this step is shown in Figure 10b. After thresholding, morphological closing and opening operations are performed to remove small black and white spots considered as noise. The kernel sizes for closing and opening are parameters that do not depend on the d-spacing and are set based on the noise characteristics of the image.
Subsequently, we perform skeletonization and branching. Skeletonization reduces the polymer regions to single-pixel-wide lines, representing the skeleton of the polymer chains. Branching breaks the skeleton at junctions where three or more connections occur, resulting in branched skeletonized representations called backbones. The output of this step is depicted in Figure 10c.
To refine the data, we proceed to segment and filter the backbones. Backbones shorter than a certain pixel length are considered noise and are filtered out; the threshold length is a parameter proportional to the d-spacing. The remaining backbones are segmented into uniform lengths, referred to as bones, for consistent evaluation. The division size is also a parameter proportional to the d-spacing. The result after filtering and division is shown in Figure 10d.
For each bone, we perform ellipse construction by fitting an ellipse to its pixel locations using the
scikit-imagelibrary [45]. This provides the major and minor axes of the bone, which are used for further analysis. Ellipses are preferred because they help filter out non-linear bones (curved structures not part of crystalline
regions) and facilitate the creation of a graph representation where each ellipse serves as a node.
To identify the crystalline regions, we apply ellipse aspect ratio filtering. Crystalline regions are composed of linear polymer backbones, so we filter out curved bones by setting a threshold on the ellipse aspect ratio (major axis length divided by minor axis length). Bones with aspect ratios above this threshold are retained for further analysis. This step’s output is shown in Figure 10e.
A graph is then constructed where each ellipse represents a node. An edge is created between two nodes if the distance between their centers is less than the adjacency distance parameter (proportional to the d-spacing) and the angle between their major axes is less than the adjacency angle parameter (also proportional to the d-spacing). The graph is stored as an adjacency matrix.
Using this graph representation, we perform clustering of adjacent bones by identifying connected components using a depth-first search algorithm. Clusters with a node count below a certain threshold are considered noise and discarded. The remaining clusters represent detected crystalline regions. For each cluster, a convex hull and an alpha shape are computed to segment the crystal region. Properties such as area, centroid, major and minor axis lengths, and orientation are evaluated. Additionally, a line indicating the direction of the crystal pattern is plotted. The output after clustering is shown in Figure 10f, and the final crystal detection is displayed in Figure 10g.
Finally, we perform d-spacing evaluation for each detected crystal region. The largest possible square region within the crystal is selected, and a Fast Fourier Transform (FFT) is performed to transform the spatial domain into the frequency domain. Band-pass filtering is applied to remove frequencies outside the range of interest. The location of the peak frequency above a set threshold is used to calculate the exact d-spacing value and the orientation of the crystal pattern. The frequency threshold is a parameter. The evaluation of d-spacing using FFT is illustrated in Figure 11. The algorithm outputs include visualizations such as convex hulls and alpha shapes of detected crystal regions overlaid on the original image, as well as quantitative data like area, centroid, major and minor axis lengths, orientation, and d-spacing values for each crystal. All features are saved in CSV files for further analysis. These outputs provide valuable insights into the material’s microstructure and can aid in understanding material properties more effectively.

Figure 11. Evaluation of d-spacing for the crystal using the Fast Fourier Transform
The algorithm is implemented in Python, utilizing the scikit-image library for image processing and SciPy for computational functions. A configuration file is used to input all relevant parameters, dataset paths, and result paths. The code is tested on Ubuntu Linux-based local systems and HPC servers. It supports batch processing and multiprocessing, allowing for efficient analysis of large datasets. Results are stored in a well-organized directory structure, with options to save intermediate outputs for debugging purposes. Comprehensive documentation is provided, detailing code usage and parameter settings. A summary of the primary and secondary parameters used in the algorithm is provided in Appendix § 10. These parameters are optimized for detecting crystals in the HRTEM dataset of PCDTBT organic photovoltaic materials.
Bayesian Optimization
Optimizing hyperparameters in image processing algorithms is crucial for enhancing performance, especially in complex tasks like automated detection of crystalline domains in HRTEM images. Traditional methods of parameter tuning, such as grid search or manual adjustment, can be inefficient and may not guarantee optimal results due to the high dimensionality and computational expense. To address this challenge, we integrated Bayesian optimization into our framework, GRATEv2, to systematically and efficiently identify the optimal set of parameters.
Bayesian optimization is a sequential design strategy for global optimization of black-box functions that are expensive to evaluate especially due to large hyperparameter space [46]. It builds a probabilistic model of the objective function and uses it to select the most promising hyperparameters to evaluate next, balancing exploration and exploitation.
Mathematical Formulation
Let denote a vector of hyperparameters, and let be the objective function that maps hyperparameters to a scalar performance metric. Our goal is to find the hyperparameters that minimize the objective function:
In our case, is the negative mean Intersection over Union (IoU) score between the detected crystalline regions and the ground truth annotations, as we aim to maximize the IoU.
Gaussian Process Surrogate Model
Bayesian optimization relies on a surrogate model to approximate the objective function. We use a Gaussian process (GP) prior [47] over functions to model . A GP is defined by its mean function and covariance function :
We assume a zero mean function without loss of generality, and select a suitable covariance function, such as the squared exponential (radial basis function) kernel:
where is the signal variance and is a diagonal matrix of length-scale parameters , controlling the smoothness of the function along each dimension.
Posterior Distribution
Given a set of observed data points , where and is Gaussian observation noise, the GP posterior predictive distribution at a new point is given by:
where , is the covariance matrix with , and .
Acquisition Function
An acquisition function guides the selection of the next evaluation point by quantifying the utility of sampling at . We use the Expected Improvement (EI) acquisition function [48], defined as:
where is the minimum observed value of the objective function. Under the GP posterior, the EI can be computed analytically:
where and are the posterior mean and standard deviation from Eqs. (6) and (7), is the standard normal cumulative distribution function, and is the standard normal probability density function.
Optimization Loop
As shown in Figure 12, the Bayesian optimization algorithm proceeds iteratively through initialization, surrogate model updates, and acquisition function optimization.

Figure 12. Flowchart of the Bayesian Optimization Loop. The process iteratively updates the surrogate model and selects new hyperparameters to evaluate until convergence criteria are met.
The Bayesian optimization algorithm begins with the initialization step, where the objective function is evaluated at an initial set of hyperparameters , often chosen via a space-filling design like Latin hypercube sampling. Next, a Gaussian Process (GP) surrogate model is fitted to the observed data , capturing our current understanding of the objective function.
Using this surrogate model, the algorithm maximizes the acquisition function to find the next promising set of hyperparameters:
The objective function is then evaluated at to obtain , and the data is augmented:
The algorithm checks if the convergence criterion is met (e.g., maximum iterations reached or negligible improvement observed). If not, the process loops back to updating the surrogate model with the new data, as depicted in Figure 12. This iterative process continues until convergence, resulting in the optimal set of hyperparameters.
Parameter Space
We optimized 13 hyperparameters of the image processing algorithm, each within specified bounds informed by prior knowledge as shown in Table 5. The ranges were chosen to cover a wide search space while ensuring that the parameters were physically meaningful and relevant to the task of detecting crystalline regions in HRTEM images.
| Hyperparameter | Range | Type |
blur_iteration | [5, 20] | Integer |
Blur_kernel_propCons | [0.1, 0.5] | Real |
closing_k_size | [1, 20] | Integer |
opening_k_size | [1, 20] | Integer |
pixThresh_propCons | [0.0, 1.0] | Real |
ellipse_len_propCons | [0.5, 5.0] | Real |
ellipse_aspect_ratio | [2.0, 7.0] | Real |
thresh_dist_propCons | [1.0, 5.0] | Real |
thresh_theta | [5.0, 15.0] | Real |
cluster_size | [1, 10] | Integer |
dspace_bandpass | [0.1, 0.5] | Real |
powSpec_peak_thresh | [1.0, 1.5] | Real |
thresh_area_factor | [1.0, 5.0] | Real |
Table 5. Hyperparameters and their ranges used in Bayesian optimization.
Objective Function Evaluation
Figure 13 shows example inputs for the Bayesian optimization training. We used a total of 13 annotated HRTEM images for training. The ground truth annotations are created using VGG annotator tool [49]. For each set of hyperparameters , the objective function was evaluated as follows:

Figure 13. Bayesian optimization training input i.e. input images, ground truth annotations, and ground truth masks. Each row represents a different sample from the dataset.
Run the image processing algorithm with parameters on the annotated HRTEM images.
Generate binary masks of the detected crystalline regions.
Compute the IoU between the detected masks and the ground truth masks:
where denotes the cardinality (number of pixels in this context).
Set the objective function value as .
Algorithm Execution
Our implementation of the Gaussian Process-based Bayesian optimization algorithm for hyperparameter tuning is based on the scikit-optimize library [50]. The library provides a user-friendly interface through the gp_minimize function for Bayesian optimization and supports various acquisition functions, surrogate models, and optimization strategies. We used the gp_minimize function with the following key parameters:
Acquisition Function: Expected Improvement (EI).
Number of Calls: 200 total evaluations of the objective function.
Initial Points: 10 random initial evaluations to seed the GP model.
Random State: Seeded for reproducibility.
Evaluation of Data Sufficiency
Determining the optimal amount of data to collect is crucial for experimentalists. Collecting too little data can compromise the reliability of results, while collecting excessive data may not yield additional insights and can waste valuable resources. Establishing a reliable stopping criterion for data collection ensures that resources are utilized efficiently without sacrificing statistical significance.
An effective data sufficiency metric should possess certain desirable features:
Sensitivity to Distribution Changes: It should accurately reflect changes in the underlying data distribution as more data is collected.
Scale Interpretability: The metric should provide a quantitative measure that is interpretable and can be related to practical thresholds for decision-making.
Applicability to Empirical Distributions: It should be suitable for comparing empirical distributions derived from finite samples.
Metric Properties: The measure should satisfy the properties of a mathematical metric, such as non-negativity, identity of indiscernibles, symmetry, and triangle inequality, to ensure consistent and meaningful comparisons.
In this study, we introduce a stopping criterion based on the Wasserstein distance to assess data sufficiency. The Wasserstein distance, also known as the Earth Mover’s Distance, quantifies the difference between two probability distributions by measuring the minimum "cost" of transforming one distribution into the other. It is particularly well-suited for our purposes because it satisfies all the desired features of a data sufficiency metric listed above.
Mathematical Definition of the Wasserstein Distance Let and be two probability distributions on the real line with cumulative distribution functions (CDFs) and , respectively. The -th order Wasserstein distance between and is defined as:
where and are the quantile functions (inverse CDFs) of and , and . For , the first-order Wasserstein distance simplifies to:
Alternatively, for discrete empirical distributions derived from finite samples, the first-order Wasserstein distance between two sets of observations and can be computed by sorting the observations and calculating the average absolute difference between the sorted values:
where , and are the ordered statistics (sorted data).
The Wasserstein distance’s ability to capture differences between distributions, even when they have overlapping support, makes it ideal for assessing the convergence of empirical data distributions as more data is collected. By monitoring the Wasserstein distance between successive data samples, we can determine when the distributions have converged, indicating that the data collection process has reached a point of diminishing returns.
Data Availability
The dataset used in this work will be made available on request.
Code Availability
The software developed for this paper is available at https://bitbucket.org/baskargroup/gratev2.
Acknowledgements
Funding from the National Science Foundation under Award DMREF 2323716 and the Office of Naval Research under Awards N00014-19-1-2453, and N00014-23-1-2001 are gratefully acknowledged.
Conflict of Interest Statement
The authors have no competing interests to declare that are relevant to the content of this article.
Appendix
Optimum Parameters
The Bayesian optimization based optimum parameters and manually selected parameters of GRATEv2 for our dataset are given in Table 6.
| Parameter | Description | Manually Selected Value | Bayesian Optimized Value |
| dspace_nm | d-Spacing in nm | 1.9 | 1.9 |
| pix_2_nm | pixels per nm | 78.5 | 78.5 |
| blur_iteration | Blurring Iterations | 15 | 20 |
| Blur_kernel_propCons | Kernel Size Blurring* | 0.15 | 0.12 |
| closing_k_size | Kernel Size Closing | 15 | 2 |
| opening_k_size | Kernel Size Opening | 17 | 2 |
| pixThresh_propCons | Threshold Pixel Length* | 0.63 | 0.74 |
| ellipse_len_propCons | Uniform Breaking Length* | 1.50 | 4.03 |
| ellipse_aspect_ratio | Threshold Ellipse Aspect Ratio | 5.00 | 4.38 |
| thresh_dist_propCons | Adjacency Distance* | 2.00 | 1.36 |
| thresh_theta | Adjacency Angle (degrees) | 10.00 | 13.96 |
| cluster_size | Threshold Cluster Size | 7 | 9 |
| dspace_bandpass | Band Pass Filter Size | 0.20 | 0.44 |
| powSpec_peak_thresh | Power Spectrum Threshold | 1.15 | 1.00 |
| thresh_area_factor | Area Threshold Factor | 4.00 | 2.79 |
Table 6. Comparison of Manually selected and Bayesian Optimized Parameters
Proportionality constant to − Spacing.
dspace_nm and pix_2_nm are user inputs and not optimized.
T-statistic for Data Sufficiency
To evaluate the statistical significance of the improvement, we performed a paired t-test on the IoU scores from the two sets of parameters. The null hypothesis () is that there is no difference in the mean IoU scores between the manual and Bayesian-optimized parameters. Let be the difference in IoU scores for each image, defined as .
The t-statistic is computed as:
With degrees of freedom, the critical t-value at a significance level of (two-tailed) is approximately 2.571. Since , we reject the null hypothesis and conclude that the improvement in IoU scores using Bayesian optimization is statistically significant.
Additional Validation Results
Figure A.1. Additional comparison of ground truth, manually selected parameters, and Bayesian-optimized parameters across different images.
References
- [1]D.B. Williams and C.B. Carter. Transmission Electron Microscopy: A Textbook for Materials Science. Number v. 1 in Cambridge library collection. Springer, 2009. ISBN 9780387765006. URL https://books.google.com/books?id=dXdrG39VtUoC.
- [2]A.G. Cullis and P.A. Midgley. Microscopy of Semiconducting Materials 2003. CRC Press, 2018. ISBN 9781351083089. URL https://books.google.com/books?id=5MUIEQAAQBAJ.DOI
- [3]R F Ergerton. Electron energy-loss spectroscopy in the tem. Reports on Progress in Physics, 72(1):016502, dec 2008. doi: 10.1088/0034-4885/72/1/016502. URL https://dx.doi.org/10.1088/0034-4885/72/1/016502.
- [4]J.C.H. Spence. High-Resolution Electron Microscopy. OUP Oxford, 2013. ISBN 9780191508400. URL https://books.google.com/books?id=PitoAgAAQBAJ.
- [5]T. Isabell, J. Brink, M. Kawasaki, B. Armbruster, I. Ishikawa, E. Okunishi, H. Sawada, Y. Okura, K. Yamazaki, T. Ishikawa, and et al. Development of a 200kv atomic resolution analytical electron microscope. Microscopy Today, 17(3):8–11, 2009. doi: 10.1017/S155192950005045.
- [6]J. Kirkland. Advanced Computing in Electron Microscopy. SpringerLink: Springer e-Books. Springer US, 2010. ISBN 9781441965332. URL https://books.google.com/books?id=YsCLlyaiNv0C.
- [7]Christoph J Brabec, Andreas Distler, Xiaoyan Du, Hans-Joachim Elgelhaaf, Jens Hauch, Thomas Heumueller, and Ning Li. Material strategies to accelerate opv technology toward a gw technology. Advanced energy materials, 10(43):2001864, 2020.DOI
- [8]Paul A Midgley and Rafal E Dunin-Borkowski. Electron tomography and holography in materials science. Nature materials, 8(4):271–280, 2009.DOI
- [9]Pal Toth, Arpad B. Palotas, Eric G. Eddings, Ross T. Whitaker, and JoAnn S. Lighty. A novel framework for the quantitative analysis of high resolution transmission electron micrographs of soot i. improved measurement of interlayer spacing. Combustion and Flame, 160(5):909–919, 2013. ISSN 0010-2180. doi: https://doi.org/10.1016/j.combustflame.2013.01.002. URL https://www.sciencedirect.com/science/article/pii/S0010218013000035.
- [10]Yihan Zhu, Jim Ciston, Bin Zheng, Xiaohe Miao, Cory Czarnik, Yichang Pan, Rachid Sougrat, Zhiping Lai, Chia-En Hsiung, Kexin Yao, et al. Unravelling surface and interfacial structures of a metal–organic framework by transmission electron microscopy. Nature materials, 16(5):532–536, 2017.DOI
- [11]A Sharma, T Kyotani, and A Tomita. A new quantitative approach for microstructural analysis of coal char using hrtem images. Fuel, 78(10):1203–1212, 1999.
- [12]Pal Toth. Nanostructure quantification of turbostratic carbon by hrtem image analysis: State of the art, biases, sensitivity and best practices. Carbon, 178:688–707, 2021.DOI
- [13]Balaji Sesha Sarath Pokuri, Jacob Stimes, Kathryn O’Hara, Michael L. Chabinyc, and Baskar Ganapathysubramanian. GRATE: a Framework and Software for Graph Based Analysis of Transmission Electron Microscopy Images of Polymer Films. Computational Materials Science, 163:1–10, 2019. ISSN 0927-0256. doi: 10.1016/j.commatsci.2019.02.030. URL https://www.sciencedirect.com/science/article/pii/S0927025619301041.DOI
- [14]Utkarsh Pratiush, Austin Houston, Sergei V Kalinin, and Gerd Duscher. Realizing smart scanning transmission electron microscopy using high performance computing. Review of Scientific Instruments, 95(10), 2024.
- [15]Ayana Ghosh, Kevin Roccapiore, Matthew G Boebinger, Debangshu Mukherjee, Anees Al-Najjar, Marshall Mcdonnell, Sergei V Kalinin, and Maxim Ziatdinov. Integrating high-performance computing with electron microscopy for scientific insights, 2024.DOI
- [16]Utkarsh Pratiush, Kevin M Roccapiore, Yongtao Liu, Sergei V Kalinin, and Gerd Duscher. Realizing smart stem via machine learning on remote high performance computer, 2024.DOI
- [17]Yael Tsarfati, Karen C Bustillo, Benjamin H Savitzky, Adam Marks, Iain McCulloch, Colin Ophus, Andrew M Minor, and Alberto Salleo. Charge-induced structural rearrangements in organic mixed ionic electronic conductors: A cryogenic 4d-stem study, 2024.DOI
- [18]Yael Tsarfati, Karen C Bustillo, Benjamin H Savitzky, Colin Ophus, Iain McCulloch, Alberto Salleo, and Andrew M Minor. Structural study of hydrated organic mixed ionic electronic conductors using cryogenic 4d-stem, 2023.
- [19]Yutong Bi, Yuan Tian, Mingjie Xu, Evgeniy Boltynuk, Leonardo Velasco Estrada, Horst Hahn, Jian Han, David J Srolovitz, and Xiaoqing Pan. In situ 4d-stem study on grain boundary dynamics in polycrystals, 2023.DOI
- [20]Robert Winkler, Alexander Zintler, Oscar Recalde-Benitez, Tianshu Jiang, Déspina Nasiou, Esmaeil Adabifiroozjaei, Philipp Schreyer, Taewook Kim, Ezter Piros, Nico Kaiser, et al. Texture transfer in dielectric layers via nanocrystalline networks: Insights from in situ 4d-stem. Nano Letters, 24(10):2998–3004, 2024.DOI
- [21]Bruno César da Silva, Zahra Sadre Momtaz, Eva Monroy, Hanako Okuno, Jean-Luc Rouviere, David Cooper, and Martien Ilse Den Hertog. Assessment of active dopants and p–n junction abruptness using in situ biased 4d-stem. Nano Letters, 22(23):9544–9550, 2022.DOI
- [22]Hsin-Yun Chao, Kartik Venkatraman, Saman Moniri, Yongjun Jiang, Xuan Tang, Sheng Dai, Wenpei Gao, Jianwei Miao, and Miaofang Chi. In situ and emerging transmission electron microscopy for catalysis research. Chemical Reviews, 123(13):8347–8394, 2023.DOI
- [23]Yael Tsarfati, Karen C Bustillo, Luke Balhorn, Tyler J Quill, Jennifer Donohue, Steven E Zeltmann, Benjamin Savitzky, Colin Ophus, Christopher J Takacs, Iain McCulloch, et al. Microstructural study of organic mixed ionic-electronic conductor thin films using 4d-stem and hrtem. Microscopy and Microanalysis, 28(S1):350–351, 2022.DOI
- [24]Nabila Masud, Jaydeep Rade, Md Hasibul Hasan Hasib, Adarsh Krishnamurthy, and Anwesha Sarkar. Machine learning approaches for improving atomic force microscopy instrumentation and data analytics. Frontiers in Physics, 12:1347648, 2024.
- [25]Maxim Ziatdinov, Ondrej Dyck, Artem Maksov, Xufan Li, Xiahan Sang, Kai Xiao, Raymond R Unocic, Rama Vasudevan, Stephen Jesse, and Sergei V Kalinin. Deep learning of atomically resolved scanning transmission electron microscopy images: chemical identification and tracking local transformations. ACS nano, 11(12):12742–12752, 2017.DOI
- [26]Catherine K. Groschner, Christina Choi, and Mary C. Scott. Machine Learning Pipeline for Segmentation and Defect Information from High-Resolution Transmission Electron Microscope Data. Microscopy and Microanalysis, 27(3):549–556, May 2021. doi: 10.1017/S1431927621000386.DOI
- [27]Jaydeep Rade, Soumik Sarkar, Anwesha Sarkar, and Adarsh Krishnamurthy. 3d reconstruction of protein complex structures using synthesized multi-view afm images. arXiv preprint arXiv:2211.4662, 2022.
- [28]Seyed Majid Azimi, Dominik Britz, Michael Engstler, Mario Fritz, and Frank Mücklich. Advanced steel microstructural classification by deep learning methods. Scientific reports, 8(1):2128, 2018.DOI
- [29]Jean-Noël Rouzaud and Christian Clinard. Quantitative high-resolution transmission electron microscopy: a promising tool for carbon materials characterization. Fuel Processing Technology, 77:229–235, 2002.
- [30]Árpád B Palotás, Lenore C Rainey, Christian J Feldermann, Adel F Sarofim, and John B Vander Sande. Soot morphology: An application of image analysis in high-resolution transmission electron microscopy. Microscopy research and technique, 33(3):266–278, 1996.pubmed.ncbi.nlm.nih.gov/8652885
- [31]Kuen Yehliu, Randy L Vander Wal, and André L Boehman. Development of an hrtem image analysis method to quantify carbon nanostructure. Combustion and flame, 158(9):1837–1851, 2011.DOI
- [32]So Hyun Park, Sungmin Park, Seungjin Lee, Jiho Kim, Hyungju Ahn, Bumjoon J. Kim, Boknam Chae, and Hae Jung Son. Development of highly efficient large area organic photovoltaic module: Effects of nonfullerene acceptor. Nano Energy, 77:105147, 2020. ISSN 2211-2855. doi: https://doi.org/10.1016/j.nanoen.2020.105147. URL https://www.sciencedirect.com/science/article/pii/S2211285520307254.
- [33]Álvaro Rodríguez-Rodríguez, Esther Rebollar, Tiberio A Ezquerra, Marta Castillejo, Jose V Garcia-Ramos, and Mari-Cruz García-Gutiérrez. Patterning of conjugated polymers by laser: Synergy of nanostructure formation in the all-polymer heterojunction p3ht/pcdtbt. Langmuir, 34(1):115–125, 2018.DOI
- [34]Zach M Bailey, Eric T Hoke, Rodrigo Noriega, Javier Dacuña, George F Burkhard, Jonathan A Bartelt, Alberto Salleo, Michael F Toney, and Michael D McGehee. Morphology-dependent trap formation in high performance polymer bulk heterojunction solar cells. Advanced Energy Materials, 1(5):954–962, 2011.DOI
- [35]Xinhui Lu, Htay Hlaing, David S Germack, Jeff Peet, Won Ho Jo, Denis Andrienko, Kurt Kremer, and Benjamin M Ocko. Bilayer order in a polycarbazole-conjugated polymer. Nature communications, 3(1):795, 2012.DOI
- [36]Ryan A Fair, Dhruv Gamdha, Joshua T Del Mundo, Abigail M Fenton, Agatha O’Connell, Karen C Bustillo, Andrew M Minor, Baskar Ganapathysubramanian, and Enrique D Gomez. Automated tem reveals intercrystalline correlations of conjugated polymers. Submitted, in review, 2025.DOI
- [37]Stefan Van der Walt, Johannes L Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D Warner, Neil Yager, Emmanuelle Gouillart, and Tony Yu. scikit-image: image processing in python. PeerJ, 2:e453, 2014.DOI
- [38]Gary Bradski, Adrian Kaehler, et al. Opencv. Dr. Dobb’s journal of software tools, 3(2), 2000.
- [39]Juan Nunez-Iglesias, Adam J Blanch, Oliver Looker, Matthew W Dixon, and Leann Tilley. A new python library to analyse skeleton images confirms malaria parasite remodelling of the red blood cell membrane skeleton. PeerJ, 6:e4312, 2018.
- [40]Pauli Virtanen, Ralf Gommers, Travis E Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, et al. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods, 17(3):261–272, 2020.DOI
- [41]Charles R Harris, K Jarrod Millman, Stéfan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, et al. Array programming with numpy. Nature, 585(7825):357–362, 2020.DOI
- [42]Renxuan Xie, Melissa P. Aplan, Nicholas J. Caggiano, Albree R. Weisen, Tang Su, Christian Müller, Mo Segad, Ralph H. Colby, and Enrique D. Gomez. Local Chain Alignment via Nematic Ordering Reduces Chain Entanglement in Conjugated Polymers. Macromolecules, 51(24):10271–10284, 2018. doi: 10.1021/acs.macromol.8b01840.DOI
- [43]Renxuan Xie, Youngmin Lee, Melissa P. Aplan, Nicholas J. Caggiano, Christian Müller, Ralph H. Colby, and Enrique D. Gomez. Glass Transition Temperature of Conjugated Polymers by Oscillatory Shear Rheometry. Macromolecules, 50(13):5146–5154, 2017. doi: 10.1021/acs.macromol.7b00712.DOI
- [44]Ryan A Fair, Renxuan Xie, Youngmin Lee, Ralph H Colby, and Enrique D Gomez. Molecular weight characterization of conjugated polymers through gel permeation chromatography and static light scattering. ACS Applied Polymer Materials, 3(9):4572–4578, 2021.DOI
- [45]Stefan Van der Walt, Johannes L Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D Warner, Neil Yager, Emmanuelle Gouillart, and Tony Yu. Scikit-image: Image processing in Python. PeerJ, 2:e453, 2014.DOI
- [46]Peter I Frazier. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811, 2018.arxiv.org/abs/1807.02811
- [47]Carl Edward Rasmussen. Gaussian processes in machine learning. In Summer school on machine learning, pages 63–71. Springer, 2003.DOI
- [48]Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13:455–492, 1998.
- [49]Abhishek Dutta and Andrew Zisserman. The VIA annotation software for images, audio and video. In Proceedings of the 27th ACM International Conference on Multimedia, MM ’19, New York, NY, USA, 2019. ACM. ISBN 978-1-4503-6889-6/19/10. doi: 10.1145/3343031.3350535. URL https://doi.org/10.1145/3343031.3350535.
- [50]T Head, M Kumar, H Nahrstaedt, G Louppe, and I Shcherbatyi. scikit-optimize/scikit-optimize (v0. 9.0). Zenodo. Available online: https://zenodo.org/record/5565057#. Y8Y-iRVBxPY (accessed on 1 July 2021), 2021.


