WorksPhysics and Astronomy
Improving conditional generative adversarial networks for inverse design of plasmonic structures
Abstract
Deep learning has emerged as a key tool for designing nanophotonic structures that manipulate light at sub-wavelength scales. We investigate how to inversely design plasmonic nanostructures using conditional generative adversarial networks. Although a conventional approach of measuring the optical properties of a given nanostructure is conceptually straightforward, inverse design remains difficult because the existence and uniqueness of an acceptable design cannot be guaranteed. Furthermore, the dimensionality of the design space is often large, and simulation-based methods become quickly intractable. Deep learning methods are well-suited to tackle this problem because they can handle effectively high-dimensional input data. We train a conditional generative adversarial network model and use it for inverse design of plasmonic nanostructures based on their extinction cross section spectra. Our main result shows that adding label projection and a novel embedding network to the conditional generative adversarial network model, improves performance in terms of error estimates and convergence speed for the training algorithm. The mean absolute error is reduced by an order of magnitude in the best case, and the training algorithm converges more than three times faster on average. This is shown for two network architectures, a simpler one using a fully connected neural network architecture, and a more complex one using convolutional layers. We pre-train a convolutional neural network and use it as surrogate model to evaluate the performance of our inverse design model. The surrogate model evaluates the extinction cross sections of the design predictions, and we show that our modifications lead to equally good or better predictions of the original design compared to a baseline model. This provides an important step towards more efficient and precise inverse design methods for optical elements.
Introduction
Many novel ways to manipulate the interaction between light and matter have been discovered during the past decades, which has been of great importance in nanophotonics research and applications[1, 2, 3]. New materials and structures, such as metamaterials and plasmonic antennas, can be designed to achieve desired optical properties and are used in a wide range of applications, including plasmonic circuits[4, 5], super resolution imaging[6] and sensing[7, 8, 9]. As the fabrication possibilities grow, so does the number of available design parameters[10]. The complexity of the design problem increases with the size of the design parameter space, leading to several challenges when developing methods to predict optimal designs with respect to the desired optical properties. These challenges include limitations in existing physics-based approaches to model light-matter interactions of particles with complex geometry, leading to a dependence on numerical simulations of electromagnetic models instead. Numerical simulations are limited by the available computational resources, which puts a constraint on both memory usage and time complexity of the design algorithm, making it a challenge to employ optimization techniques that handle large amounts of design parameters[11, 12]. Therefore we need smarter, more efficient approaches for inverse design, and in this work we develop a machine learning framework for this purpose. Various machine learning approaches have been tested for nanophotonic inverse design applications[10, 11, 13, 14]. A particularly effective machine learning approach has been to use deep learning models based on neural networks to automate the design process. The main advantage of the deep learning approach is that it provides versatile methods that allows for learning arbitrary, non-linear continuous functions between finite-dimensional sets of inputs and outputs. This is possible since neural networks act as universal function approximators in the sense that, there exist a neural network to approximate any continuous function to a desired accuracy. Formally, any Borel measurable function can be approximated with a neural network given that there are enough hidden units available in the layers of the network[15]. Furthermore, deep learning models excel at handling high-dimensional data, and there is a broad range of applications, including image analysis and non-linear regression tasks that are highly relevant for nanophotonics[16]. Therefore, it is clear that this family of methods are well for the inverse design problem in nanophotonics[17].
Using a data-driven approach, the inverse design models are optimized using synthetic data generated from electromagnetic simulations of the problem at hand. Promising implementations include convolutional neural networks (CNN) as forward models predicting the optical response of a plasmonic particle[18, 19], a tandem deep neural network (DNN) combining the forward model with an inverse network for predicting designs[20], conditional autoencoder (cAE) models[21, 22], and conditional generative adversarial networks (cGAN)[23, 24, 25, 26]. The forward model approach is usually combined with another optimization technique, such as evolutionary optimization algorithms. In this case, the forward model acts as a surrogate model for ultra-fast evaluation[27, 28]. In contrast, tandem DNN, cAE, and cGAN models are examples of direct inverse design modeling using neural networks. With this approach, the main challenge to address is the non-uniqueness of solutions called the one-to-many problem. Since many inverse design problems lack a unique solution, there may exist several solutions or even no solution to a given spectral input[29].
While many deep learning techniques have been applied to improve design rules in nanophotonics, the focus has been mostly on finding a model that works for the particular application of interest, and not on the optimization. Hence, this work focuses on how to improve existing models typically used by the nanophotonics community. In particular, we study the conditional GAN model to design plasmonic nanostructures of anisotropic shape, including plasmonic dimers. For such structures, the optical response depends on the polarization of the incoming light[30], making them relevant for studying plasmonic effects in the near-field. We make two significant optimizations to the model that increase its accuracy and its convergence rate. The improvements are shown for two different network architectures. One simple feed-forward neural network architecture consisting of fully connected layers, that we call the fully connected GAN model (FCGAN)[31]. Second, we propose a more complex deep convolutional GAN model (DCGAN) introduced by Radford, Metz, and Chintala[32] and used by So and Rho[23], for the inverse design of silver antennas. We also apply Wasserstein GAN (WGAN) loss and training
algorithm to stabilize the optimization that is inherently unstable, often leading to convergence issues[33]. Figure 1 presents an overview of our inverse design framework based on the cGAN-model. The goal of cGAN-model is to learn a generator function that maps from a stochastic input space to the design space conditioned on the desired optical spectra. Given the spectra and the stochastic vector it outputs a prediction of the optimal design . During training, a critic network provides feedback to the generator by estimating the statistical distance between the distribution of original and predicted design images, and the model is optimized to minimize this quantity.

Figure 1. This schematic presents an overview of the inverse design framework presented in this work. a) We train a cGAN-model consisting of a generator and a critic network, to predict optimal designs of dimer structures. Based on the desired cross-section spectra , and a stochastic vector , the generator learns to predict the design . The critic network provides feedback to the generator by estimating the statistical distance between the original design and produced by the generator. The model is empirically optimized to minimize the statistical distance between samples, using simulation data. Training is stopped when the distribution of has converged to the distribution of . b) We pre-train a CNN based forward model to predict the spectra associated with a generated design and use it for performance evaluation. The forward model allows for estimating the mean absolute error (MAE) between the spectra of the generated design and the original spectra, providing a quantitative measure of their similarity. In addition, we evaluate the image based MAE of the predicted design as well.
We use a surrogate-model-based metric to evaluate the performance of the cGAN-model. The surrogate model uses a pre-trained forward model based on a CNN-architecture, which predicts the spectra associated with a given design image. This allows for estimating the mean absolute error (MAE) in the spectra by using the forward model to predict the spectra of both the original and generated designs. In addition to the surrogate model, we also use an image-based performance metric. The MAE is evaluated for the actual design images, i.e. between and , to evaluate the pixel to pixel loss in the predicted designs.
Methods
Conditional GAN-models have commonly been used for inverse design problems in nanophotonics[23, 24]. Specifically, the cGAN model is used to learn the relationship between the geometry of a structure and its optical properties. We use the scattering and absorption cross section spectra as the conditional input. The model consists of two networks, a generator , and a discriminator , that are trained together to gradually improve their performance[34]. In general, the inverse design problem does not have unique solution, but cGAN-models handle this well because the generator model takes a stochastic input as well, making it possible to learn several solutions to a single optical spectra. For this reason, the cGAN-model have become a popular choice when inversely designing nanostructures to satisfy the users desired optical properties.
In our study, we utilize the cGAN-model to generate nanostructure geometries , conditioned on the structures scattering and absorption cross-section spectra . The goal is to learn a generator model to mimic the training data distribution , from taking a stochastic vector , conditioned on the label vector , as model input. Furthermore, we use the Wasserstein loss function to optimize our model, as suggested by Gulrajani et al. 33. In this case the discriminator is called the critic since it measures the statistical distance between true and generated distributions .
Our model architecture is illustrated in Figure 2. We use an improved method to input the conditional data into the critic model, based on the label projection method proposed by[35]. The method introduces the conditional data into the model with an inner product, instead of concatenation or vector addition. This projection-based method is more consistent with the underlying probabilistic model of the standard adversarial discriminator loss[34] and Miyato and Koyama 35 show that it improves the generation of class conditional images. Thus, we assume that similar performance gains can be achieved when using the Wasserstein loss function[33]. In addition to label projection, we also add capacity in the critic using a label embedding network to further improve its ability to learn features from the conditional input. The label embedding network consists of multiple one-dimensional convolutional layers to process the input data, and the full cGAN-model is mathematically formulated as
Figure 2. The general architecture of the conditional GAN model consists of critic network and generator network. The critic network is used by the Wasserstein GAN training algorithm in the optimization process. During the optimization process, the generator network is trained to generate images that replicate the probability distribution of the training data. Both networks takes conditional label data as input. The label data is the absorption and scattering cross section spectral data of the corresponding training images used to train the model. Proposed in this work is to introduce the conditional label data using label projection in the critic network. The label vector is projected onto the feature vector of the image processing network (blue) with an inner product and this is called label projection. Furthermore we also propose the use of a label embedding network to encode the label data. The label embedding network uses several layers of the one-dimensional convolutions.
where
is the critic model, and
is the generator model. Both are parametrized with neural networks and optimized with respect to eq. (4). The critic model takes an image and spectral data as input, and outputs a scalar. It contains the image downsampling network , and the embedding network , which downsample the spectral data into a lower dimensional latent space. The dimension of the latent space is chosen as . Table I describes the architecture of the embedding network. Batch normalisation is used after each convolutional layer, together with an activation function. For consistency with the downsampling networks of the two architectures, we use ReLU as activation in the FCGAN-model, and LeakyReLU with a slope of 0.2 in the DCGAN-model. The second term in eq. (2) is the label projection term, where the conditional information is introduced with a euclidean inner product. A small multilayer perceptron (MLP) is also used to make the dimension compatible with the label projection term. The generator takes a Gaussian vector , where size of is 100, and spectral data as input and outputs an image . The generator upsampling network is denoted by . During the development of our model, we found that including the label embedding network in the generator improved the performance of our models. This is a logical consequence, as the additional model parameters allows the generator to learn a richer set of features from the input spectra, leading to more precise predictions of nanostructure geometries. Therefore, the embedding network was included in the generator architecture as well. For the critic and the generator, we study two different neural network architectures. First, we show that a simple non-convolutional model using fully connected layers exclusively, benefit from our proposed improvements. We call this the FCGAN-model, where both the critic and the generator have 3 hidden layers with 512 neurons each, and a ReLU activation function after each hidden layer. Second, we also study our model on the deep convolutional architecture proposed by[32] and used in the inverse design study by So and Rho 23. The network architectures details can be found in the Supplementary Material.
| Stage | Operation | Channels | Kernel /Stride /Padding |
| 1 | Conv1D + BatchNorm1D + Activation | 32 | 5/2/0 |
| 2 | Conv1D + BatchNorm1D + Activation | 64 | 5/2/2 |
| 3 | Conv1D + BatchNorm1D + Activation | 128 | 5/2/2 |
| 4 | Conv1D + BatchNorm1D + Activation | 512 | 3/2/1 |
| 5 | Average Pool | - | - |
| Input dim | Output dim | ||
| 6 | FC-layer | 512 |
Table I. The table summarize the label embedding network used in the GAN-models.
Section VI.
We use the Wasserstein distance in the loss function with a gradient penalty term, as it has been shown to provide smoother gradients that stabilize the optimisation process[33]. The loss is summarized as
where the critic and the generator are parametrized with neural networks. Gulrajani et al.33 enforce a soft version gradient penalty term, where is an interpolation between a real and generated sample. The hyperparameter controls the size of the penalty term, and we use in our study, as proposed by Gulrajani et al.33. Our model is optimized with the proposed training algorithm from[33].
Results and Discussion
Our proposed cGAN models were trained on a dataset consisting of anisotropically shaped plasmonic nanostructures. The dataset includes both dimers and structures of elliptical shape. The structures are made of gold and placed on top of a glass substrate, which defines the domain of size 500x500 nm in cross-section. When creating the dataset, the aim was to vary the available geometrical parameters, while simultaneously limiting any possible bias. Therefore, we varied the height, diameter and dimer gap using uniform distributions to randomly generate different dimers. The polarization of the light was always 45° to excite both the parallel and the perpendicular axis of the dimers[30]. We used a steady state simulation of each generated structure using the finite element method (FEM) and calculated the absorption and scattering cross-sections. The COMSOL Multiphysics® software and its wave optics module was used to perform the simulations[36]. In the dataset, the dimer gap ranges from 5-40 nm, the long axis length scale from 40-100 nm, while the height scale is 14-146 nm. We employ the Brendel-Bormann model[37] for the permittivity of gold to simulate the electromagnetic response of the structures. All structures are simulated for wavelengths in the range 400-800 nm, using 16 nm steps. In total, the dataset contained 2898 samples of different structures with the corresponding spectral data.
We used the Wasserstein GAN training algorithm[33] to train the models for 30000 epochs on the dimer dataset, and both the FCGAN and the DCGAN architectures are used in the critic and the generator networks. The effect of adding label projection and the embedding network is evaluated in a comparison study. Figure 4 shows the progression of the validation error during training, calculated on a separate validation set not used by the optimisation algorithm. From the validation set, we estimated the MAE in two different ways and use it as our metric for model performance. For the first method, we calculated the MAE between the cGAN-models output, and the original image corresponding to the label used as input. In the second method, we used a pre-trained convolutional regression model to predict the spectral data of a nanostructure image. This approach allows us to estimate the label error instead of the pixel error, by calculating the MAE on the output from the regression model. To estimate the mean value of the given metric, we drew stochastic samples from the cGAN-model and considered the average.

Figure 4. The models were trained on a dataset containing cylindrical dimer structures and they were evaluated using two different methods. The first method uses a pre-trained convolutional neural network regression model to predict the scattering cross section and absorption cross section spectra. This approach allows us to estimate the MAE between the spectra predicted from real and GAN-generated images. As a second evaluation method, we also estimate the mean absolute error in pixel values between real and generated images. Figures a) and c) shows the results from training the FCGAN architectures figures b) and d) shows the results from training the DCGAN-architecture. We calculate the evaluation metrics by generating sample images for each labeled original image, and take the average of those when computing the error estimate.
Figure 3 presents an example prediction from the FCGAN-model on test data not seen at all during the model development. The FEM simulation data in fig. 3d and fig. 3e are the conditional inputs used to generate the output displayed in fig. 3b and fig. 3c. We used the pre-trained regression model to evaluate the model’s performance, by predicting corresponding spectral data of the output images. The evaluated spectral data are plotted together with the FEM-simulated data in fig. 3d and fig. 3e. Adding label projection and the embedding network to the model improves image quality, and consequently, the estimated spectral data lie closer to the FEM-simulated data. This is especially true for the scattering cross section in this specific example.

Figure 3. The results in this figure are obtained from the FCGAN-model with and without using label projection and an embedding network in the critic. Figure a) show the original image and figures b) and c) shows the images predicted by the models. Figures d) and e) show the scattering and absorption cross section spectras corresponding to the images. The blue curve is the original data from the finite element simulation of the gold nano structure in the original image. We use a pre-trained convolutional neural network regression model to predict the spectral data for both the original image (dashed red) and the images generated by the model (dashed green and orange). The quality is visibly worse for the image generated from FCGAN compared to when we add label projection and the embedding network. This is also causing a pertubation in the spectral data prediction.
From the results in fig. 4, we find for both network architectures that the convergence time for training the model decreases when label projection is added. We believe the label projection helps the optimization of the model parameters as shown by[35], leading to fewer epochs required for convergence. Adding the embedding network, also speeds up the convergence, but the effect is smaller compared to adding label projection. This results in faster training of the models which is important for such computationally heavy algorithms. It should be noted that the embedding network increases the training time per epoch by approximately 17%, compared to only using the label projection. This has to be considered when selecting the best version of the model as the embedding network slows down training, but in some cases improves error estimates.
We also find that the performance, in terms of error estimates, improves for the FCGAN-model. The improvement is present, both for adding label projection and the embedding network, but the combination of the two results in the smallest error. This suggests that the model needs more capacity since it benefits from the additional parameters added in the embedding network. For the DCGAN-model, the model performance reaches what looks as a lower limit for the model on this dataset. The error converges to roughly the same value for the 4 variants, which indicates that there is enough capacity in the models for learning from this dataset. However, the DCGAN-model + LP + Embed. converges in about 5000 epochs, which is more than 3 times faster compared to the standard DCGAN-model. All models in the study were trained for 30000 epochs, and this leads to over-fitting for the variants that converge faster. This is expected as they reach their optimum earlier, which results in over-training in the remaining epochs.
In addition, the FCGAN-models were trained on another dataset, containing structures with a more complex shape. Figure 5 illustrates predictions on test data for three versions of the FCGAN-model. These results demonstrate that our models can also learn from datasets with more complex shapes as well, and all the predictions improved by adding the LP + Embed. into the model.

Figure 5. This figure shows example images from training the FCGAN-models on a larger dataset containing differently shaped anisotropic gold structures. The first row contains the original images, and the other rows contains the corresponding predicted images by the FCGAN-models. The model predictions improve when label projection and the embedding network are added to the model.
It is clear from the training results on the cylindrical dimer dataset in Figure 4, that both the FCGAN and DCGAN model benefit from our proposed improvements. The DCGAN-model sees a significant decrease in convergence time as label projection is added to the critic architecture, and adding the embedding network improves even more the performance. However, the models produce similar error estimates across the four versions. This indicates that the deep convolutional architecture in itself has a sufficient number of trainable parameters to capture the feature of the dataset, leading to a near-optimal solution in each case.
Notably, for the FCGAN-model, we found that the combination of using the label embedding network with the label projection gives the best performance, in terms of the number of epochs required for convergence and error estimates. These results are well illustrated in Figure 5 and Figure 3 where predictions from the FCGAN-model are shown. The image quality improves significantly as the label projection is added, leading to a better estimate of the optimal nanostructure design. This is an important finding, as it shows that a simple model without convolutional layers can be improved with small modifications to a low cost in computational resources. Compared to the more computationally expensive DCGAN model, its performance is not that far off in terms of ererror estimates, particularly when considering corresponding cross-section spectra in Figure 4c and Figure 4d. In practice, training would have been stopped before the models starts to overfit, ensuring the lowest possible error estimate is achieved. Therefore, a fair comparison should evaluate the two models at their respective optimal error estimates, that is the epoch where the lowest error is achieved. In doing so, we conclude that the best FCGAN model is performing well in terms of the MAE on cross-section spectra compared to the best DCGAN model. However, the DCGAN models are slightly better when it comes to the MAE of the actual images, since the convolutional layers in the DCGAN model helps to generate higher quality images.
Challenges and Limitations
With a growing design parameter space, there may exist more than one solution to the conditional input provided. In general, one cross-section spectra may correspond to several structures in the design space, and this fact is illustrated in Figure 6. The examples are chosen to illustrate the non-uniqueness of solutions to the inverse design problem, which makes the problem particularly challenging when designing an inverse design model. Several solutions may exist for a single input, possibly leading to parts of the solution space not being reachable by the model[38]. Conditional GAN models are well suited to tackle this problem as the stochastic input allows multiple solutions to be learned, and as seen in Figure 6, the FCGAN-model with label projection and embedding network, successfully learns to output structures with a similar cross-sectional spectra as the original one. In many cases the output is a 180 degree rotation of the original structure, due to the symmetry of the problem with respect to the polarisation vector of the linearly polarised light illuminating the structures. This is fine since parity symmetry is not violated, but still gives an image different from the original one. Other predictions include structures where the output shape does not match exactly the original one, but the cross-sections spectra are still similar which is what we want to achieve. The model fails to make accurate predictions for some inputs, typically the smaller structures where the image resolution potentially starts to become a limiting factor. Even in this case the main optical features are still captured. It seems that the size of the predicted structures as well as the gap between the dimers, are consistently well estimated, and this leads to reasonably small perturbations in the cross-sections spectra for the examples in Figure 6.

Figure 6. The figure shows images generated from the FCGAN + LP + Embedding network model and the corresponding spectral data predicted by a pretrained regression model, just as in Figure 3. Original images are to the left, and generated ones are to the right. The samples are chosen to illustrate the fact that one specific cross section spectra might correspond to several nanostructures designs, the so called one-to-many problem. As a result, the GAN-model output can differ from the original image while its corresponding spectra are still closely related. For instance, the model outputs images that are mirrored compared to the original image. This is likely explained by the symmetry of the structures with respect to the polarization vector, that makes a 45 degree angle with the long axis of the dimers. Images c), and k) illustrate the phenomenon. We also observe cases where the model outputs structures with where the shape does not match exactly. Often the cross section spectra is still similar, as seen from a), b), d), e), h), i), j).
The use of the Wasserstein GAN training algorithm and loss function should also be discussed as it has its own strengths and weaknesses. Most importantly, it has a stabilizing effect on the optimization process since the loss function provides smoother gradients, which avoids instability problems such as vanishing and exploding gradients. Those problems are preferably avoided when optimizing GAN-models, since the convergence of the algorithm become very sensitive to the hyperparameter values, particularly the learning rate. Using the Wasserstein loss comes at the cost though, because the computational load increases, as the penalty term in (4) requires an additional gradient computation in each iteration, which leads to slightly longer training time.
Conclusion
Conditional generative adversarial networks have recently emerged as popular inverse design models within the nanophotonic community. In this study, we have shown how two standard models can be modified to improve the convergence of the training algorithm and the accuracy of the predicted results. Most importantly, these results are achieved through pure algorithmic improvements without adding a large amount of new parameters to the model architecture. This keeps the cost of optimizing the models down, both in terms of training and memory usage. We believe this is an important step towards more efficient inverse design models for nanophotonic applications.
Code and Data availability
The code and all data that support the findings in this study are publicly available in the git repository https://github.com/pettper/Improving-conditional-generative-adversarial-networks-for-inverse-design-of-plasmonic-structures
Acknowledgments
This work had been funded by Swedish Research Council (Grant No. 2021-05784), the Knut and Alice Wallenberg Foundation through the Wallenberg Academy Fellows Program (Grant No. 2023.0089) and the European Research Council through the ERC Starting Grant ‘MagneticTWIST’ (Grant No. 101116253).
We thank the High Performance Computing Center North (HPC2N) at Umeå University for providing computational resources and valuable support during test and performance runs.
References
Supplementary Material
Figures S.2 and S.1 show schematics of the neural networks used in the proposed inverse design model for downsampling and upsampling of images. In the DCGAN-model we use the convolutional architecture proposed by Radford, Metz, and Chintala[32] modified to suite the label projection used in our models. This architecture utilizes the two-dimensional convolutional operation for downsampling and the corresponding transposed convolutional operation for upsampling. In the DCGAN downsampling block, we use instance normalization[39] instead of batch normalization, to avoid introducing correlations between samples. This makes the gradient penalty work properly as suggested by as suggested by[33]. The LeakyReLU activation function with a slope of 0.2 is applied after all hidden layers in the downsampling block. In the DCGAN upsampling block we use batch normalization[40] together with ReLU as activation function after the transposed convolutions in the hidden layers. In the output layer, the tangent hyperbolic function is used as activation function as in[32] which requires the images to be normalized to the range [-1, 1]. The details of the layers are showed in table S.1 and table S.2 for images.
Figure S.1. The figure shows the FCGAN and DCGAN downsampling blocks.
In the FCGAN-model we use fully connected layers with a ReLU activation function between each layer. The hidden layers contain 512 neurons each. The upsampling and downsampling blocks are similar in their internal and differ only in the input and output layers. The tangent hyperbolic function is used as output activation in the upsampling block, just as in the DCGAN-model. The details of the layers are showed in table S.3 and table S.4 for images. Figure S.3 shows the validation error from our metrics after training the four different versions of the FCGAN-model. We use a dataset containing anisotropic gold nano structures and train the models for
Figure S.2. The figure shows the FCGAN and DCGAN architectures for the image upsampling blocks.
| Stage | Operation | Channels in → out | Kernel /Stride /Padding | Image Size in → out |
| 1 | Conv + LeakyReLU(0.2) | 2 → 64 | 4/2/1 | |
| 2 | Conv + InstanceNorm + LeakyReLU(0.2) | 64 → 128 | 4/2/1 | |
| 3 | Conv + InstanceNorm + LeakyReLU(0.2) | 128 → 256 | 4/2/1 | |
| 4 | Conv + InstanceNorm + LeakyReLU(0.2) | 256 → 512 | 4/2/1 | |
| 5 | Conv + InstanceNorm + LeakyReLU(0.2) | 512 → 1024 | 4/2/1 | |
| 6 | Conv | 1024 → 50 | 4/2/0 |
Table S.1. The table summarize the critic architecture for the DCGAN model.
| Stage | Operation | Channels in → out | Kernel /Stride /Padding | Image Size in → out |
| 1 | ConvTranspose + BatchNorm + ReLU | 1024 → 512 | 4/2/1 | |
| 2 | ConvTranspose + BatchNorm + ReLU | 512 → 256 | 4/2/1 | |
| 3 | ConvTranspose + BatchNorm + ReLU | 256 → 128 | 4/2/1 | |
| 4 | ConvTranspose + BatchNorm + ReLU | 128 → 64 | 4/2/1 | |
| 5 | ConvTranspose | 64 → 2 | 4/2/1 | |
| 6 | Tanh | 2 → 2 | – |
Table S.2. The table summarize the generator architecture for the DCGAN model.
15000 epochs.
The DCGAN-model was also trained on the dataset with anisotropic gold nano structures and fig. S.4 shows one example prediction after training for 30000 epochs. We see that there is excellent agreement with the original image both with and without label projection. The estimation of the spectral data, also show very good agreement with the original simulation data.
| Stage | Operation | Input dim | Output dim |
| 1 | FC-Layer + ReLU | 32768 | 512 |
| 2 | FC-Layer + ReLU | 512 | 512 |
| 3 | FC-Layer + ReLU | 512 | 512 |
| 4 | FC-Layer + ReLU | 512 | 512 |
| 5 | FC-Layer | 512 |
Table S.3. The table summarize the critic network for the FCGAN-model.
| Stage | Operation | Input dim | Output dim |
| 1 | FC-Layer + ReLU | 512 | |
| 2 | FC-Layer + ReLU | 512 | 512 |
| 3 | FC-Layer + ReLU | 512 | 512 |
| 4 | FC-Layer + ReLU | 512 | 512 |
| 5 | FC-Layer | 512 | 32768 |
| 6 | Tanh | 32768 | 32768 |
Table S.4. The table summarize the generator network for the FCGAN-model.

Figure S.3. The models were trained on a dataset containing anisotropic gold nano structures and they were evaluated using two different methods. The first method uses a pre-trained convolutional neural network regression model to predict the scattering cross section and absorption cross section spectra. This approach allows us to estimate the mean absolute error between the spectra predicted from real and GAN-generated images. As a second evaluation method, we also estimate the mean absolute error in pixel values between real and generated images. Figures a) and b) shows the results from training the FCGAN architectures. We calculate the evaluation metrics by generating sample images for each labeled original image, and take the average of those when computing the error estimate.

Figure S.4. The results in this figure are obtained from the DCGAN-model with and without using label projection and an embedding network in the critic. Figure a) show the original image and figures b) and c) shows the images predicted by the models. Figures d) and e) show the scattering and absorption cross section spectras corresponding to the images. The blue curve is the original data from the finite element simulation of the gold nano structure in the original image. We use a pretrained convolutional neural network regression model to predict the spectral data for both the original image (dashed red) and the images generated by the model (dashed orange and green). The results are similar in terms of image quality and estimated cross section spectrum for both variants of the model, and the predicted designs agree well with the original structure.
References
- [1]A. González-Tudela, A. Reiserer, J. J. García-Ripoll, and F. J. García-Vidal, “Light–matter interactions in quantum nanophotonic devices,” Nature Reviews Physics 6, 166–179 (2024).
- [2]N. Maccaferri, G. Barbillón, A. N. Koya, G. P. Acuna, and D. Garoli, “Recent advances in plasmonic nanocavities for single-molecule spectroscopy,” Nanoscale Advances 3, 633–642 (2021).DOI
- [3]A. N. Koya, M. Romonelli, J. Kuttruff, N. Henriksson, A. Stefancu, G. Grinblat, A. De Andrés, F. Schnurr, M. Vanzan, M. Marsili, et al., “Advances in ultrafast plasmonics,” Applied Physics Reviews 10 (2023).
- [4]T. J. Davis, D. E. Gómez, and A. Roberts, “Plasmonic circuits for manipulating optical information,” Nanophotonics 6, 543–559 (2017).DOI
- [5]A. Tuniz, O. Bickerton, F. J. Diaz, T. Käsebier, E.-B. Kley, S. Kroker, S. Palomba, and C. M. de Sterke, “Modular nonlinear hybrid plasmonic circuit,” Nature Communications 11, 2413 (2020).
- [6]K. A. Willets, A. J. Wilson, V. Sundaresan, and P. B. Joshi, “Super-resolution imaging and plasmonics,” Chemical Reviews 117, 7538–7582 (2017), pMID: 28084729.DOI
- [7]M. Mesch, B. Metzger, M. Hentschel, and H. Giessen, “Nonlinear plasmonic sensing,” Nano Letters 16, 3155–3159 (2016), pMID: 27050296.DOI
- [8]K. M. Mayer and J. H. Hafner, “Localized surface plasmon resonance sensors,” Chemical Reviews 111, 3828–3857 (2011), pMID: 21648956.DOI
- [9]J. Langer, D. Jimenez de Aberasturi, J. Aizpurua, R. A. Alvarez-Puebla, B. Auguié, J. J. Baumberg, G. C. Bazan, S. E. J. Bell, A. Boisen, A. G. Brolo, J. Choo, D. Cialla-May, V. Deckert, L. Fabris, K. Faulds, F. J. García de Abajo, R. Goodacre, D. Graham, A. J. Haes, C. L. Haynes, C. Huck, T. Itoh, M. Käll, J. Kneipp, N. A. Kotov, H. Kuang, E. C. Le Ru, H. K. Lee, J.-F. Li, X. Y. Ling, S. A. Maier, T. Mayerhöfer, M. Mosk ovits, K. Murakoshi, J.-M. Nam, S. Nie, Y. Ozaki, I. Pastoriza-Santos, J. Perez-Juste, J. Popp, A. Pucci, S. Reich, B. Ren, G. C. Schatz, T. Shegai, S. Schlücker, L.-L. Tay, K. G. Thomas, Z.-Q. Tian, R. P. Van Duyne, T. Vo-Dinh, Y. Wang, K. A. Willets, C. Xu, H. Xu, Y. Xu, Y. S. Yamamoto, B. Zhao, and L. M. Liz-Marzán, “Present and future of surface-enhanced raman scattering,” ACS Nano 14, 28–117 (2020), pMID: 31478375.
- [10]K. A. Brown, S. Brittman, N. Maccaferri, D. Jariwala, and U. Celano, “Machine learning in nanoscience: big data at small scales,” Nano Letters 20, 2–10 (2019).DOI
- [11]S. Molesky, Z. Lin, A. Y. Piggott, W. Jin, J. Vucković, and A. W. Rodriguez, “Inverse design in nanophotonics,” Nature Photonics 12, 659–670 (2018).
- [12]Z. Ma, Z. Liu, Z. A. Kudyshev, A. Boltasseva, W. Cai, and Y. Liu, “Deep learning for the design of photonic structures,” Nature Photonics 15, 77–90 (2021).
- [13]S. Moon, J. Kang, and J.-S. Yeo, “Review of generative models for the inverse design of nanophotonic metasurfaces,” Applied Science and Convergence Technology 32, 141–150 (2023).
- [14]J.-F. Masson, J. S. Biggins, and E. Ringe, “Machine learning for nanoplasmonics,” Nature Nanotechnology 18, 111–123 (2023).DOI
- [15]K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural networks 2, 359–366 (1989).
- [16]Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature 521, 436–444 (2015).DOI
- [17]S. So, T. Badloe, J. Noh, J. Bravo-Abad, and J. Rho, “Deep learning enabled inverse design in nanophotonics,” Nanophotonics 9, 1041–1057 (2020).
- [18]I. Sajdaian, J. Kim, and J. Rho, “Finding the optical properties of plasmonic structures by image processing using a combination of convolutional neural networks and recurrent neural networks,” Microsystems & nanoengineering 5, 27 (2019).
- [19]S. Noureen, M. Zubair, M. Ali, and M. Q. Mehmood, “Deep learning based hybrid sequence modeling for optical response retrieval in metasurfaces for stpv applications,” Optical Materials Express 11, 3178–3193 (2021).DOI
- [20]D. Liu, Y. Tan, E. Khoram, and Z. Yu, “Training deep neural networks for the inverse design of nanophotonic structures,” Acs Photonics 5, 1365–1369 (2018).
- [21]W. Ma, F. Cheng, Y. Xu, Q. Wen, and Y. Liu, “Probabilistic representation and inverse design of metamaterials based on a deep generative model with semi-supervised learning strategy,” Advanced Materials 31, 1901111 (2019).
- [22]W. Ma and Y. Liu, “A data-efficient self-supervised deep learning model for design and characterization of nanophotonic structures,” Science China Physics, Mechanics & Astronomy 63, 284212 (2020).
- [23]S. So and J. Rho, “Designing nanophotonic structures using conditional deep convolutional generative adversarial networks,” Nanophotonics 8, 1255–1261 (2019).
- [24]S. An, B. Zheng, H. Tang, M. Y. Shalaginov, L. Zhou, H. Li, M. Kang, K. A. Richardson, T. Gu, J. Hu, et al., “Multifunctional metasurface design with a generative adversarial network,” Advanced Optical Materials 9, 2001433 (2021).
- [25]Z. Liu, D. Zhu, S. P. Rodrigues, K.-T. Lee, and W. Cai, “Generative model for the inverse design of metasurfaces,” Nano letters 18, 6570–6576 (2018).
- [26]J. Jiang, D. Sell, S. Hoyer, J. Hickey, J. Yang, and J. A. Fan, “Free-form diffractive metagrating design based on generative adversarial networks,” ACS nano 13, 8872–8878 (2019).
- [27]T. Chugh, C. Sun, H. Wang, and Y. Jin, “Surrogate-assisted evolutionary optimization of large problems,” in High-Performance Simulation-Based Optimization (Springer, 2019) pp. 165–187.
- [28]T. Feichtner, O. Selig, and B. Hecht, “Plasmonic nanoantenna design and fabrication based on evolutionary optimization,” Optics express 25, 10828–10842 (2017).
- [29]P. R. Wiecha, A. Arbouet, C. Girard, and O. L. Muskens, “Deep learning in nano-photonics: inverse design and beyond,” Photonics Research 9, B182–B200 (2021).DOI
- [30]R. Verre, N. Maccaferri, K. Fleischer, M. Svendsen dahl, N. O. Länk, A. Dmitriev, P. Vavassori, I. V. Shvets, and M. Käll, “Polarization conversion-based molecular sensing using anisotropic plasmonic metasurfaces,” Nanoscale 8, 10576–10581 (2016).
- [31]M. H. Sazli, “A brief review of feed-forward neural networks,” Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering 50 (2006).
- [32]A. A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434 (2015).
- [33]I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville, “Improved training of wasserstein GANs,” Advances in neural information processing systems 30 (2017).arxiv.org/abs/1704.00028
- [34]I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in neural information processing systems 27 (2014).
- [35]T. Miyato and M. Koyama, “cGANs with projection discriminator,” arXiv preprint arXiv:1802.05637 (2018).arxiv.org/abs/1802.05637
- [36]COMSOL AB, “COMSOL Multiphysics® v. 6.3,” https://www.comsol.com (2025), stockholm, Sweden.
- [37]A. D. Rakić, A. B. Djurišić, J. M. Elazar, and M. L. Majewski, “Optical properties of metallic films for vertical-cavity optoelectronic devices,” Applied optics 37, 5271–5283 (1998).
- [38]K. Dai, K. Sun, X. Yan, O. L. Muskens, C. de Groot, X. Zhu, Y. Hu, H. Duan, and R. Huang, “Inverse design of structural color: finding multiple solutions via conditional generative adversarial networks,” Nanophotonics 11, 3057–3069 (2022).DOI
- [39]D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Instance normalization: The missing ingredient for fast stylization,” arXiv preprint arXiv:1607.08022 (2016).
- [40]S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International conference on machine learning (pmlr, 2015) pp. 448–456.


