Fluorescence microscopy remains one of the most powerful tools for imaging cell biology samples because of its ability to specifically label molecular structures and to visualize them in different fluorescence color channels. It also offers exceptionally high sensitivity and can visualize molecular processes even below the optical diffraction limit. Several methods that enable imaging of fluorescently labeled samples down to the nanoscale have been developed during the last two decades [1]. Super-resolution structured illumination microscopy (SR-SIM) is a particularly compelling method, because it works with the majority of samples and fluorophores commonly used in cell biology without imposing specific requirements on sample preparation [2,3]. Therefore, it can even be applied to living samples [4–12]. SR-SIM in its most common form uses a series of sinusoidal illumination patterns with a pattern periodicity at or near the diffraction limit. This patterned excitation light is phase-shifted laterally and rotated to different discrete angles to acquire a series of raw images, which are then passed on to an image reconstruction algorithm to obtain the final super-resolved SR-SIM image [13]. Several implementations of reconstruction algorithms that operate in the frequency space were developed and a number of open access tools are now available that aim to enhance the speed and spatial resolution, and minimize reconstruction artifacts in the final reconstructed image (i.e., fairSIM [14], OpenSIM [15], SIMToolbox [16], and CC-SIM [17]). The most common property among all these image reconstruction algorithms is that they require a series of high-quality raw images to be able to reconstruct a high-quality super-resolution image. However, reconstruction in the frequency space fails to reliably reconstruct SR-SIM images if the signal-to-noise ratio (SNR) is too low (e.g., because the laser excitation power level was too low), the sample exposure time chosen was too short, or the sample has already undergone irreversible photobleaching [18]. Recently, several developments that aim to reduce reconstruction artifacts from SR-SIM images have been undertaken. Huang et al. used a fully analytical approach to reduce noise and minimize reconstruction artifacts using the Hessian matrix theory [10]. Hoffman and Betzig proposed the reconstruction of SIM images in lower pixel count tiles with subsequent merger to reduce reconstruction artifacts [19]. Jin et al., on the other hand, used deep neural networks to reconstruct the cropped regions of SR-SIM images [20]. Christensen et al. used a deep learning architecture for the reconstruction from synthetic raw SIM images [21] with subsequent testing on real microscope images. Ling et al. relied on a special type of convolutional neural network, a CycleGAN, for the same purpose [22]. And Weigert et al. used deep learning algorithms to enhance isotropic resolution and the signal-to-noise ratio (SNR) of fluorescence microscopy images in general [23].

Here, we provide a comprehensive study on several deep-learning based approaches to denoise and reconstruct SR-SIM images. First, as starting point, we include an end-to-end deep learning architecture and workflow that is related to the existing literature combining SIM and deep learning [20–22] in that a single network is used that receives raw SIM images as input and produces a super-resolution image as the output. This approach is named super-resolution residual encoder–decoder structured illumination microscopy (SR-REDSIM) in the following discussion. In this case, the entire SIM reconstruction process is performed by the deep convolutional neural network.

Second, for what we believe, to the best of our knowledge, is an entirely novel workflow, we combine classical computational SIM reconstruction with a deep learning network. This workflow is called a residual encoder–decoder fairSIM (RED-fairSIM). RED-fairSIM is a combination of the fairSIM image reconstruction package [14], which performs image reconstruction using commonly used frequency domain algorithms and a deep convolutional neural network for subsequent artifact reduction and denoising. Finally, we also show the results for a workflow where deep learning is first applied to the raw SIM images for denoising with subsequent classical computational SIM reconstruction; thus, exactly the other way around, as in RED-fairSIM. This approach is named preRED-fairSIM.

For the main network architecture for all three approaches we use the residual encoder–decoder network (RED-Net) [24], which is comparatively lightweight. For training, raw image data directly from the microscope is used, which offers a straightforward, practical solution without the need to create synthetic data or carry out substantive preprocessing in contrast to the existing literature. We found that the first two methods are robust in their ability to significantly improve the quality of the reconstructed SR-SIM images. We show, for what we believe is the first time, that the trained networks of these two methods generalize well to real microscopic SIM data with different SNRs. The novel RED-fairSIM workflow shows superior performance in this regard. Furthermore, we also demonstrate that RED-fairSIM generalizes better even if the microscope settings are changed after training.

Deep learning methods rely on training data, which, in our case, consist of noisy raw SIM images for the input and ideally noise and artifact free, super-resolved SIM reconstructions for the output. Thus, we first need to generate such a data set that is large enough to effectively train the network, but also captures all aspects of the SIM imaging process (sample behavior, instrument imperfections, and data processing artifacts) well enough.

In principle, the data acquisition process of a SIM microscope can be simulated. In this case, the expected output represents the ground truth data upon which the simulation is based, and which is known without SNR or resolution limits. In addition, the amount of available training data would only be limited by the processing time, as the generation would be fully automated and not rely on access to a microscope system. However, we decided against this pure in silico approach. Although the basic effects of structured illumination, Poisson-distributed noise, and even basic optical imperfections are rather easy to simulate, modeling the response of a full structured illumination microscope correctly is very complex. Additionally, such a simulation would likely have to be adjusted to reflect the properties of a specific SR-SIM instrument to capture changes (e.g., when switching to a different manufacturer or even a specific installation of an SR-SIM microscope). The same argument holds true for the fluorescent samples themselves. Although some simulations that provide perfect ground-truth data exist (e.g., for single molecule localization microscopy [25,26]), they, again, do not capture all of the variability found in real-world samples.

The option chosen for data generation for the work presented here is to use real microscope data from standard biological samples. This approach naturally captures all aspects and imperfections of the samples and of the specific instrument in question [23], but also poses constraints. Because data collection requires both instrument time and manual sample preparation and handling, the amount of training data is naturally limited. There also is no perfect ground truth available. To acquire the high-quality reference images presented to the networks as desired output, we adjusted the instrument to provide high-SNR raw frames and processed those with the classical, frequency-domain-based image reconstruction algorithm. While these images are low in noise and reconstruction artifacts, they are never completely devoid of them. For this reason, we refer to them by the term “reference image” instead of “ground truth.” To acquire noisy, low-SNR images as input, the samples were then photo-bleached by continued exposure and acquisition of raw SIM images, which naturally reduces the fluorescent light output over time and results in a series of images with steadily decreasing SNR. (See Section 2.C for details.)

U2OS cells were cultured in DMEM supplemented with 10% FBS and grown on round coverslips of 170±5nm thickness (No. 1.5H). Cells were fixed with 4% PFA for 15 min, followed by PBS washes, and permeabilization with 0.5% Triton-X100 for 3 min. Another two rounds of PBS washes were done prior to blocking with 3% BSA. For the immunolabeling, the microtubuli, cells were stained with anti-tubulin Ab (Invitrogen Cat. No. 322500) 1:400 for 2 h at room temperature, followed by a PBS wash and one additional hour of incubation with Alexa 488-conjugated anti-mouse IgG 1:400. Cells were then briefly washed with PBS before Vectashield was applied to embed the coverslip onto the standard slide glass for imaging.

The DeltaVision|OMX (GE Healthcare, Chicago, IL, USA) was used to acquire 3D-SIM raw images. A total of 101 randomly selected fields of view were acquired by exposing each field-of-view to the full laser power of the 488 nm excitation laser and the exposure time was set at 20 ms for each of the 15 raw image frames. A total of 200 image repetitions were collected at each position without delays. Taking into account the camera readout and pattern switching time, acquiring 15 raw SIM images, making up one timestamp in the raw image stack, takes approximately 375 ms.

In our work, we used 101 different cell structures (fields of view). Out of these, 81 were selected for the training data and the remaining 20 for the test data. Each cell structure was captured for 200 repetitions. During each repetition, 15 frames were recorded, iterating the phase and orientation of the sinusoidal SIM illumination pattern, yielding an image stack of size 15×512×512 (frames, width, height). During this time-lapse acquisition, the samples underwent photobleaching, which reduces the amount of active fluorescent emitters and thus the amount of emitted photons. Therefore, less and less light is captured and the SNR steadily decreases during the acquisition of such a time series.

The cell structures from the timestamps 175 to 200 are therefore considered as noisy training and test input while the samples from timestamp 0 are considered as clean output images. All 15 clean raw SIM images of the 101 cell structures from timestamp 0 are used to reconstruct high-resolution reference SIM images of size 1024×1024 pixels by using fairSIM, which employs a classic frequency-domain-based reconstruction. In this work, the input dimension is 15×512×512 (frames, width, height) whereas the output dimension is 1024×1024 (width, height) pixels. A total of 2525 samples were further divided into training and test data. The training data contains 2025 images of the first 81 cell structures; the test data is composed of 500 test images that are created from the remaining 20 cell structures.

The only preprocessing step involved in our work is the linear scaling of the training and test data to match the overall brightness between the input and output. In addition, we tested an image augmentation approach to double the amount of training data by rotating each image by an angle of 180°.

The data from each time series over 200 repetitions were subdivided into different noise levels. Noise level 0 stands for the highest SNR in our data at timestamp 0. In our work, this is our reference data. The image data from timestamps 175–200 represent the highest noise level 4, the data from timestamps 125–150 represent noise level 3, the data from timestamps 75–100 are noise level 2, and the data from timestamps 25–50 are noise level 1. In this study, data from noise level 4 are only used in the training process whereas data from noise levels 1, 2, 3, and 4 are used in the test phase.

In the first deep-learning based SR-SIM image reconstruction method, named SR-REDSIM, the reconstruction and denoising of noisy raw SIM images are both performed by a single deep learning model. This model is a modified version of the RED-Net. RED-Net is an encoding–decoding framework with symmetric convolutional–deconvolutional layers along with skip-layer connections. It was previously used to accomplish different image restoration tasks such as image denoising, image super-resolution, and image inpainting [24]. The original RED-Net architecture is only composed of encoding–decoding blocks with the size of the network input being the same as the size of the network output. Therefore, super-resolution with this architecture has to rely on explicit image pre-upsampling [24]. In contrast, our modified RED-Net architecture contains an additional upsampling block after the encoding–decoding blocks. This upsampling block inside our model has the advantage that the input images are first denoised in their lower-dimensional space, which reduces the training time and effort.

Most of the super-resolution architectures such as the enhanced deep super-resolution network (EDSR) [27] or the residual channel attention network (RCAN) [28] are very deep and require a significant amount of training data. In comparison, our architecture is comparably lightweight.

In Eq. (1), yi represents the true pixel intensity, y^i represents the predicted pixel intensity, and n is the number of pixels (in the image).

The RED-fairSIM is combination of fairSIM and RED-Net. The pipeline of RED-fairSIM is shown in Fig. 1(b). In this approach, first fairSIM is used to transform the raw SIM images into a super-resolved output image by employing a classic, frequency-domain SIM reconstruction algorithm. This output image now contains noise, which, due to the frequency-domain algorithm, takes a SIM-specific form, and might show other reconstruction artifacts. It is subsequently processed by RED-Net.

The fairSIM reconstruction is performed in three steps: parameter estimation, reconstruction, and filtering. Mathematical and algorithmic details are provided in the original publication [14]. A synthetic optical transfer function, with NA=1.4, λ=525nm, and a=0.31 (a is a compensation parameter [14,29]), is used. A total of 500 counts of background are subtracted per pixel. SIM reconstruction parameters (pattern orientation and global phase, for example) are automatically determined by the fairSIM standard in an iterative cross-correlation approach. Filter parameters are set to a generalized Wiener filter strength of w=0.05, apodization is set at 1.9× the resolution limit with a bend of 0.8, and a notch-style filter is implemented as OTF attenuation with a strength of 0.995 and an FWHM of 1.2μm−1 is in use. For the detailed meaning and influence of these parameters, please refer to the fairSIM source code, its accompanying publication [14], and this general guide to SIM reconstruction parameters [29].

The architecture of RED-Net used in RED-fairSIM consists of 15 convolutional and 15 deconvolutional layers along with symmetric skip connections, as shown in Fig. 2(b). The output of fairSIM is propagated into the RED-Net to denoise the reconstructed noisy sample. During the training phase, the noisy SR-SIM images of size 1024×1024 pixels along with the reference SR-SIM images of the same size are used as input–output pairs for the RED-Net. The network is trained for 100 epochs with the ADAM optimizer and L2 loss.

U-Net is also a popular deep learning architecture that is extensively used in the domain of image restoration such as image denoising and super-resolution [23,30,31]. In U-Net-fairSIM, we simply replaced the RED-Net from the RED-fairSIM approach with the U-Net architecture. The U-Net is also trained for 100 epochs with the ADAM optimizer and L2 loss.

One of the well-known open-source reconstruction algorithm implementations is fairSIM. It is widely used for super-resolution tasks among the other tools in SIM microscopy. However, it cannot reconstruct a clean, high-quality super-resolution image from noisy raw SIM images. In the RED-fairSIM method, fairSIM is first used to reconstruct the SIM samples of noisy raw SIM images and then RED-Net is used to denoise the output of fairSIM and to generate high-quality super-resolution images. During the reconstruction process, the stack of 15 noisy images (three angles, five phases) of size 512×512 pixels is again propagated into the fairSIM reconstruction algorithm that then generates a single noisy reconstructed image of size 1024×1024 pixels. The noise and artifacts found in these images do not follow a typical distribution (e.g., Poisson or Gaussian), but have a distinct form that comes from the frequency-based reconstruction algorithm. This single noisy reconstructed image is further passed into the RED-Net architecture to achieve the final result, which can be seen in Fig. 3 (column 4). The complete pipeline of the RED-fairSIM method can be seen in Fig. 1(b) and the architecture of RED-Net is shown in Fig. 2(b). The parameters that were used to generate the SIM reconstructed samples from the raw SIM images are explained in Section 2.E.1. RED-Net was trained in a supervised way where the input–output pairs contain the noisy and reference reconstructed images.

The performance of this method on the unseen test samples is the best among our experiments with respect to PSNR and SSIM values, as shown in Table 1, as well as visually. The ROIs in Fig. 3 show clearly that the output images generated by RED-fairSIM are of high quality with fine details and smooth lines. They are superior compared to the noisy fairSIM, the reference fairSIM, and the SR-REDSIM. Even the artifacts introduced by fairSIM in the reference images are completely removed by RED-fairSIM. It might appear as if the contrast in the denoised SR-REDSIM and RED-fairSIM images in Fig. 3 is weaker when compared to the reference image. This, however, is not the case. The images shown in Fig. 3 are original image data as produced by the various denoising or reconstruction methods without further image processing (i.e. contrast adjustments). Part of the apparent higher contrast in the reference image can also be attributed to graininess of the image, which is caused by uneven antibody staining of the tubulin filaments (a typical phenomenon of antibody staining). It should be noted that the DL-based denoising methods also (at least partly) remove this unevenness.

In the pipeline proposed for RED-fairSIM, we replaced the RED-Net by U-Net and analyzed the resulting super-resolution images. We named this approach as U-Net-fairSIM. Figure 3 (column 3) shows that U-Net-fairSIM also produces better results as compared to the noisy, reference, and SR-REDSIM images. However, it does not surpass RED-fairSIM. Similarly, the U-Net-fairSIM approach outperforms all other counterparts except for RED-fairSIM concerning the PSNR and SSIM values in Table 1. Comparing RED-fairSIM and U-Net-fairSIM directly, as in Fig. 5, the cell structures reconstructed by RED-fairSIM are smoother. Furthermore, they are more faithful when taking the reference as “gold standard” into account. For these reasons, we have focused the presentation in this paper on RED-fairSIM.

During all of the reported experiments, we did not carry out any preprocessing on the input or output images such as downsampling or cropping, as discussed in Section 2.C. Nonetheless, we tested whether or not image augmentation yields any significant improvement in the results. For this purpose, each image was rotated by an angle of 180° and added in this form to the training set. This increased the amount of training data from 2025 to 4050 images. The mean PSNR and SSIM values after training the methods with the augmentation approach are reported in Table 2. While the image augmentation did not provide a noticeable advantage overall and did not change the performance-wise ordering of the proposed methods, and instead doubled the training time and increased the preprocessing effort, we decided to focus this paper on the results without image augmentation.Table 2.

Mean PSNR and SSIM Values along with Standard Deviations (STD) for Noise Level 4 after Applying Data Augmentation (Computed on 500 Test Images)

The results of transfer learning and fine-tuning are quite promising in terms of the achieved SSIM and PSNR values. The model trained with data of phase 0 and orientation 0 has a mean PSNR = 33.22 dB and a mean SSIM = 0.90 on the test data. Averaged over all other combinations of phase and orientation, transfer learning yields a PSNR = 31.03 dB and an SSIM = 0.89 during testing. After fine-tuning the first and last five layers of the pretrained model, the mean values over all combinations of phase and orientation amount to a PSNR = 33.32 dB and an SSIM = 0.89. These numbers suggest that the denoising of the raw SIM images works well for all orientation phases.

BM3D is a conventional state-of-the-art image denoising method from the field of computer vision [35]. During this work, we also used BM3D to denoise the noisy super-resolution images reconstructed by fairSIM. BM3D is able to remove the noise successfully from reconstructed noisy SIM samples, as shown in column 3 of Fig. 8, but fails to recover lost information.

In addition to visual inspection, quantitative results are given in Table 1, which contains the mean PSNR and SSIM values of 500 test inputs from each noise level. Here, U-Net-fairSIM is included in the comparison. Both RED-fairSIM and U-Net-fairSIM show a gradual decrease in PSNR and SSIM values from noise level 1 (weak noise) to noise level 4 (strong noise). SR-REDSIM performs similarly, but with noticeably smaller PSNR and SSIM values. The results of fairSIM without denoising deteriorate quickly when moving to higher noise levels. The most important takeaway from Table 1 and Fig. 9 is that the networks—although trained for a specific high noise level—generalize well to conditions with a better SNR.

The reconstruction of the lower noise level 0 through SR-REDSIM and RED-fairSIM highlights a second use case. At this noise level, the SNR of the raw frames is high enough to provide the reference data sets, which, as discussed before, are of high quality, but still feature some SIM reconstruction artifacts. Those artifacts are successfully removed by both SR-REDSIM and RED-fairSIM. A reasonable assumption is that, like noise, reconstruction artifacts are random enough in nature, so they are not picked up by the network during training, and thus cannot be reproduced. This effect is well known in applications such as Noise2Noise [36], where the inability of a neural network to learn random (noisy) data is explicitly used for denoising.

The results of this study provide sufficient evidence that SR-REDSIM and RED-fairSIM can both be employed to denoise and reconstruct high-quality SR-SIM images. In contrast, preRED-fairSIM in its current form is not suitable for this purpose because the output of RED-Net, although successfully denoised, contains additional artifacts noticeable in Fourier space, which spoil the performance of the subsequent classical computational SIM reconstruction. We also investigated the robustness of the successful methods (SR-REDSIM and RED-fairSIM) and showed that high-quality reconstruction of SIM samples is possibly irrespective of the noise level in the raw SIM images. The SR-REDSIM and RED-fairSIM methods outperform their counterparts, as shown in Figs. 3 and 4. Furthermore, these approaches are useful even in the absence of clean ground-truth data, as we have shown especially for RED-fairSIM where the reference data used for training contains many reconstruction artifacts. We have also shown in column of Fig. 9 that the proposed methods SR-REDSIM and RED-fairSIM can be used to remove the reconstruction artifacts from the reference image after training, so even if high SNR data can be acquired easily, SR-REDSIM and RED-fairSIM still offer an improvement over the classical reconstruction approaches.

A recent study [22] used cycle-consistent generative adversarial networks (CycleGANs) [37] to reconstruct SR-SIM images by using three to nine clean raw SIM images. A CycleGAN contains two generators and two discriminators with multiple losses that are trained in a competitive process. Therefore, CycleGANs are generally very difficult to train. Furthermore, the authors did not address the challenge of denoising. Christensen et al. [21] trained deep neural networks by using synthetic data instead of real microscope SIM images to reconstruct SR-SIM images. Although the synthetic data used in their studies for training is unrelated to real microscopes, they were successful in generating output comparable to computational tools like fairSIM. However, they did not use real noisy microscope data to test the denoising performance of their networks, and their approach was also not completely successful in the case of (simulated) high-level noise. Jin et al. [20] used multiple concatenated U-Nets to reconstruct SR-SIM images by using three to 15 raw SIM images. They trained their models on cropped and resized SIM samples and manually discarded tiles with only background information. These preprocessing steps are time-consuming, and the training of two adjacent U-Net models is also computationally expensive.

Our proposed methods use raw SIM images in their original size, which does not involve any major preprocessing steps. The amount of training data used, about 100 fields of view for training and test data together, is also small enough that specific training, capturing both a given instrument and a specific biological structure of interest, should often be feasible. While SR-REDSIM has similarities to other proposed end-to-end deep learning approaches for SIM [20–22], to the best of our knowledge, RED-fairSIM is a completely novel deep learning approach for SIM which is, as our data shows, superior to SR-REDSIM.

While both SR-REDSIM and RED-fairSIM provide high-quality reconstruction, an obvious difference between them is their ability to generalize to different SIM imaging settings. As an initial test, we varied the spatial frequency of the SIM pattern [using a 642 nm (instead of 488 nm) excitation light], which commonly happens when designing experiments and choosing dyes, as shown in Fig. 4. We then performed reconstruction with RED-fairSIM and SR-REDSIM, both trained on the original 488 nm data. Here, the RED-fairSIM approach, where the change in spatial frequency of the pattern is absorbed by the classic reconstruction step, still works very well in suppressing noise and SIM artifacts. SR-REDSIM, on the other hand, where the SIM pattern has been learned by the network, created heavy ghosting artifacts. While further validation and cross-testing are needed, this suggests that RED-fairSIM should be able to generalize to different SIM microscopes, excitation wavelengths, and probably illumination types (three-beam, two-beam, TIRF-SIM), while SR-REDSIM would require retraining whenever larger changes in these parameters occur.

Besides visual impression, the quantitative measures chosen for our comparisons are PSNR and SSIM. As a microscopy technique, obviously spatial resolution estimates would present another desirable parameter. Both Fourier ring correlation (FRC) [38,39] and image decorrelation analysis (IDA) [40] are typically chosen for this task, because they offer a quantitative resolution estimate that is not dependent on manual measurement of single structural features, which easily introduces bias. However, much care is needed to correctly apply these methods, as they are based on strict assumptions about the input data, regarding the statistically independent and spatially uncorrelated distribution of noise. Even slight correlation, introduced, for example, by camera readout electronics [41], will yield an overestimation of spatial resolution. We have performed FRC analysis and IDA on the datasets presented here, and while some results seem reasonable, others are clearly unphysical. As we cannot assume that the denoised output fulfills the assumptions of FRC or IDA, we would argue against using unmodified FRC or IDA to estimate resolution in these images. We believe further research into robust resolution estimation that works more independently of the data generation process is needed.

We also compared our set of deep-learning based approaches to classical denoising algorithms, with BM3D as a general approach and Hessian SIM as a noise filter tailored to ideally time-lapsed SIM data. Both of the classical algorithms do not reach the denoising performance of the deep learning methods. However, these algorithms do not require any prior knowledge, in the form of training data, of the sample. Thus, in contrast to deep learning methods, they can be applied to arbitrary, unknown structures. They can also serve as a cross-check, if concerns arise that the deep-learning based approaches generate artifacts stemming from their training data.

In this work, we presented two different methods, SR-REDSIM and RED-fairSIM, to reconstruct super-resolution SIM images from raw SIM images with low SNR. We demonstrated that these methods are robust against different noise intensities and do not need any retraining or fine-tuning even if the SNR is varied between the training and application. However, the generalization ability of RED-fairSIM under different SIM imaging conditions (i.e., changed microscope settings) is superior compared to SR-REDSIM. This shows that the combination of fairSIM for reconstruction and RED-Net for denoising is more promising than an end-to-end deep learning approach like SR-REDSIM. Both methods are particularly useful for SIM images with a low SNR since the traditional reconstruction algorithms cannot denoise and generate reconstruction artifacts in the SR-SIM images. Both of our proposed methods can remove these reconstruction artifacts. The overall results also show that our methods outperform other classical denoising methods like BM3D to denoise the noisy SR-SIM images. Furthermore, the proposed methods can potentially be used in the future to handle live-cell SIM imaging data as well as the reconstruction and denoising of SIM images with low SNR from other biological structures.

Acknowledgment. The authors would like to thank Dr. Matthias Fricke and David Pelkmann from the Center for Applied Data Science (CfADS) at Bielefeld University of Applied Sciences for providing access to the GPU compute cluster and Dr. Wolfgang Hübner for providing U2OS cell data with a red fluorescent tubulin stain. We are also thankful to Wadhah Zai el Amri (CfADS) for precursory work on deep learning for the denoising of microscope images. The authors would additionally like to thank Drs. Florian Jug, Carlas Smith, Peter Dedecker, and Wim Vandenberg for fruitful discussions on the application of deep learning to SIM reconstruction. During this work, we used multiple Nvidia Tesla P100 graphics cards. Each network is trained in parallel with two Tesla P100 to reduce the computational time. The machine learning code is written based on the Python libraries keras and tensorflow, whereas the fairSIM code is written in Java. The code and data used during this work can be provided upon request. The generated results are available at https://doi.org/10.5281/zenodo.4134841.