Journals > > Topics > Microscopy
Microscopy|40 Article(s)
Large-Field Microscopic Imaging Method Based on Cycle Generative Adversarial Networks
Rao Fu, Yu Fang, Yong Yang, Dong Xiang, and Xiaojing Wu
ObjectiveIn traditional optical microscopes, large field of view and high resolution cannot be achieved at the same time. Large numerical aperture objective lens is necessary to obtain high-resolution images, which will inevitably lead to the reduced imaging field of view. Similarly, a small numerical aperture objective lens should be employed to obtain a large imaging field of view, and the corresponding imaging resolution will inevitably decrease. Therefore, when the traditional optical microscope is adopted, a trade-off is needed between imaging resolution and imaging field of view, and it is impossible to obtain large field of view and high-resolution microscopic images simultaneously. With the rapid development of deep learning technology, different algorithm models are designed based on deep learning, and the corresponding pathological cell models are diagnosed in combination with the evaluation and guidance of professional doctors to yield computer-aided calculation and detection effects. Therefore, this paper proposes a method to generate matching high-resolution virtual stained images from low-resolution stained images with large field of view using cycle generative adversarial networks (Cycle-GANs), without changing the optical systems and detector devices. Only ordinary microscopes are employed to obtain staining images with large field of view, and it is unnecessary to convert the multiple objectives to obtain high-resolution images. Virtual high-resolution staining images at any position of the large field of view can be predicted through network model calculation, and the high resolution and large field of view can be realized at the same time.MethodsIn this paper, Cycle-GANs are employed to complete the experiment. Firstly, the data set preparation is carried out, and the unpaired and paired data sets of images under multiple objectives are collected respectively to constitute the training and testing parts of the network model. Secondly, a theoretical pre-experiment is conducted. The resolution of the same stained onion epidermal cell images is scaled to simulate the reduction of image resolution. The resolution of the input images is verified to have a lower limit, and the field of view of the microscope cannot be infinite in actual observation. Then, through imaging and testing the resolution testing board, the experiment of generating 25 times high-resolution virtual images from 10 times low-resolution images is carried out, which proves that Cycle-GANs networks can improve the image resolution. Finally, experiments are carried out to generate 25 times high-resolution virtual staining images corresponding to 10 times and 4 times low-resolution real staining images using stained onion epidermal cells. The corresponding values and results are obtained through subjective vision and objective evaluation indicators.Results and DiscussionsIn the experiment of grading and scaling the resolution of the same stained onion epidermal cell images (Fig. 4), the fitting curves of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) decrease with the reducing image resolution, while those of normalized root mean square error (NRMSE) increase with the decreasing image resolution (Fig. 5). With the resolution reduced by 32 times, the SSIM value lower than 0.5, and the PSNR value lower than 20, the images can no longer be correctly resolved by human eyes, which proves that the image resolution cannot be infinitely reduced. Then, in the experiment of generating 25 times high-resolution virtual images from 10 times low-resolution images on a resolution testing board (Fig. 6), low-resolution images under 10 times objective in the second row in Fig. 6 are generated by the algorithm as shown in the third row in Fig. 6. The sets of the fourth column and fifth column of the resolution testing board can be well resolved by human eyes, which shows the improvement of image resolution. Finally, in the experiment of generating 25 times high-resolution virtual stained images corresponding to 10 times low-resolution real stained images of onion epidermal cells (Fig. 7), the maximum SSIM, the maximum PSNR, and the minimum NRMSE are 0.684, 23.523 dB, and 0.131, respectively. In the experiment of generating 25 times high-resolution virtual images from 4 times low-resolution images (Fig. 8), the SSIM and PSNR can be as high as 0.597 and 22.322 dB, and the NRMSE is as low as 0.180. Under the analysis of subjective vision or objective evaluation indicators, the results of virtual stained images generated based on Cycle-GANs are close to those of real bright-field stained images under the measurement of various indicators.ConclusionsTraditional optical microscopes cannot combine large field of view with high resolution. Therefore, this paper proposes a method to generate matching high-resolution virtual stained images from low-resolution stained images with large field of view by Cycle-GANs. This can transform low-resolution stained images with low-power bright field into matching high-power virtual stained high-resolution images. Experimental results show that the characteristic information, such as organelle structures in the bright-field stained images of onion epidermis cells, can be extracted effectively by Cycle-GANs. Finally, virtual cell staining is realized, and staining damage to the tissues is reduced. Additionally, the large field of view and high resolution of the traditional optical microscope can be considered at the same time by inputting low-resolution stained images with a large field of view and outputting high-resolution virtual stained images with a high similarity to high-resolution real stained images with a small field of view. Finally, the differences between the generated virtual stained images and real bright-field stained images are evaluated by subjective vision and objective evaluation indicators. The results show that the virtual stained images generated by Cycle-GANs and real stained images are highly similar under the measurement of various indicators. Therefore, this method can meet the large field of view and high resolution of traditional microscopes. When applied to real life, it can reduce the costs and the burden of doctors and is of great significance in biomedical cell research. ObjectiveIn traditional optical microscopes, large field of view and high resolution cannot be achieved at the same time. Large numerical aperture objective lens is necessary to obtain high-resolution images, which will inevitably lead to the reduced imaging field of view. Similarly, a small numerical aperture objective lens should be employed to obtain a large imaging field of view, and the corresponding imaging resolution will inevitably decrease. Therefore, when the traditional optical microscope is adopted, a trade-off is needed between imaging resolution and imaging field of view, and it is impossible to obtain large field of view and high-resolution microscopic images simultaneously. With the rapid development of deep learning technology, different algorithm models are designed based on deep learning, and the corresponding pathological cell models are diagnosed in combination with the evaluation and guidance of professional doctors to yield computer-aided calculation and detection effects. Therefore, this paper proposes a method to generate matching high-resolution virtual stained images from low-resolution stained images with large field of view using cycle generative adversarial networks (Cycle-GANs), without changing the optical systems and detector devices. Only ordinary microscopes are employed to obtain staining images with large field of view, and it is unnecessary to convert the multiple objectives to obtain high-resolution images. Virtual high-resolution staining images at any position of the large field of view can be predicted through network model calculation, and the high resolution and large field of view can be realized at the same time.MethodsIn this paper, Cycle-GANs are employed to complete the experiment. Firstly, the data set preparation is carried out, and the unpaired and paired data sets of images under multiple objectives are collected respectively to constitute the training and testing parts of the network model. Secondly, a theoretical pre-experiment is conducted. The resolution of the same stained onion epidermal cell images is scaled to simulate the reduction of image resolution. The resolution of the input images is verified to have a lower limit, and the field of view of the microscope cannot be infinite in actual observation. Then, through imaging and testing the resolution testing board, the experiment of generating 25 times high-resolution virtual images from 10 times low-resolution images is carried out, which proves that Cycle-GANs networks can improve the image resolution. Finally, experiments are carried out to generate 25 times high-resolution virtual staining images corresponding to 10 times and 4 times low-resolution real staining images using stained onion epidermal cells. The corresponding values and results are obtained through subjective vision and objective evaluation indicators.Results and DiscussionsIn the experiment of grading and scaling the resolution of the same stained onion epidermal cell images (Fig. 4), the fitting curves of structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) decrease with the reducing image resolution, while those of normalized root mean square error (NRMSE) increase with the decreasing image resolution (Fig. 5). With the resolution reduced by 32 times, the SSIM value lower than 0.5, and the PSNR value lower than 20, the images can no longer be correctly resolved by human eyes, which proves that the image resolution cannot be infinitely reduced. Then, in the experiment of generating 25 times high-resolution virtual images from 10 times low-resolution images on a resolution testing board (Fig. 6), low-resolution images under 10 times objective in the second row in Fig. 6 are generated by the algorithm as shown in the third row in Fig. 6. The sets of the fourth column and fifth column of the resolution testing board can be well resolved by human eyes, which shows the improvement of image resolution. Finally, in the experiment of generating 25 times high-resolution virtual stained images corresponding to 10 times low-resolution real stained images of onion epidermal cells (Fig. 7), the maximum SSIM, the maximum PSNR, and the minimum NRMSE are 0.684, 23.523 dB, and 0.131, respectively. In the experiment of generating 25 times high-resolution virtual images from 4 times low-resolution images (Fig. 8), the SSIM and PSNR can be as high as 0.597 and 22.322 dB, and the NRMSE is as low as 0.180. Under the analysis of subjective vision or objective evaluation indicators, the results of virtual stained images generated based on Cycle-GANs are close to those of real bright-field stained images under the measurement of various indicators.ConclusionsTraditional optical microscopes cannot combine large field of view with high resolution. Therefore, this paper proposes a method to generate matching high-resolution virtual stained images from low-resolution stained images with large field of view by Cycle-GANs. This can transform low-resolution stained images with low-power bright field into matching high-power virtual stained high-resolution images. Experimental results show that the characteristic information, such as organelle structures in the bright-field stained images of onion epidermis cells, can be extracted effectively by Cycle-GANs. Finally, virtual cell staining is realized, and staining damage to the tissues is reduced. Additionally, the large field of view and high resolution of the traditional optical microscope can be considered at the same time by inputting low-resolution stained images with a large field of view and outputting high-resolution virtual stained images with a high similarity to high-resolution real stained images with a small field of view. Finally, the differences between the generated virtual stained images and real bright-field stained images are evaluated by subjective vision and objective evaluation indicators. The results show that the virtual stained images generated by Cycle-GANs and real stained images are highly similar under the measurement of various indicators. Therefore, this method can meet the large field of view and high resolution of traditional microscopes. When applied to real life, it can reduce the costs and the burden of doctors and is of great significance in biomedical cell research.
Acta Optica Sinica
- Publication Date: Mar. 10, 2023
- Vol. 43, Issue 5, 0518002 (2023)
Fourier Ptychography Super-Resolution Imaging Based on Square Aperture
Junyuan Liu, Haoming Shan, and Xiangsheng Xie
ObjectiveFourier ptychography (FP) is a recently developed phase retrieval approach, which simultaneously realizes image restoration with super-resolution and a large field of view. This approach is also a novel super-resolution optical imaging technique that can be efficiently applied to microscopy and macroscopy respectively. However, one limitation of FP is the long acquisition and reconstruction time due to the numerous low-resolution (LR) images that are needed. Especially when the LR image is acquired under large angle illumination (dark field imaging), the low illumination intensity leads to the long exposure time of the imaging sensors. The situation becomes more time-consuming when a new sample is imaged, and thus a larger number of images with different exposure time for each LR image should be tested to obtain the corresponding best exposure time. Losing low gray value pixels exists in the FP algorithm preprocessing, which ultimately leads to the loss of image information due to the weak local image intensity in the recovery result. With the development of optical integration and micro/nano fabrication technology, the circular lens is no longer necessary. On the contrary, the square lens can fill the entire aperture of the miniaturized imaging system (cuboid) and collect more light fields. This paper introduces the square aperture lens into the FP approach which is conducive to collecting the complete aperture light field for further miniaturized imaging devices. The introduced square lens is expected to improve the image intensity and resolution of the FP approach, reduce the data acquisition time, and be more in line with the optical integration development.MethodsAccording to the Rayleigh criterion, the low resolution of the optical imaging system is caused by the lens aperture. The FP approach achieves super-resolution and large-field imaging by combining multiple LR images. Different from the FP microscopy, the macroscopic FP technique requires moving the imaging camera or using array imaging cameras to obtain a series of LR images of the object. Each LR image is obtained by locating the aperture on the spatial spectrum plane with different positions set in horizontal and vertical directions. After the LR image is acquired at the current position, the aperture is moved to the next position according to the required step size until the spatial spectrum is fully collected. To overcome the position offset of each LR image, the template matching method based on cross-correlation is applied for image registration. This paper studies the macroscopic FP imaging technology based on square aperture. Using the imaging sensor and its natural rectangular structure in the spatial Fourier domain is more effective to use the synthesis of square aperture to improve the imaging resolution and reduce the time consumption. The macroscopic FP technology based on circular aperture is also tested to make a better comparison. In the simulation, the ptychographical iterative engine (PIE) algorithm on the MATLAB platform is adopted to iteratively recover the phase to obtain a high-resolution image. In the experiment, a macroscopic FP optical setup is built. A square aperture, a lens, and a CCD are mounted on an optical bench as a comment imaging system. The system translated in the x-y direction to collect different spatial spectra and obtain a series of LR images. The PIE algorithm used in the simulation can recover the super-resolution image from the LR images.Results and DiscussionsWhen the circular aperture is used in the FP method, the image intensity is too weak as luminous flux lacks. The low gray value pixels would be missed during the FP algorithm preprocessing, which ultimately leads to image information loss due to the weak local image intensity in the recovery result. Compared to the circular aperture, the square aperture has a greater luminous flux and can obtain higher image intensity when collecting image information in the spatial spectrum plane. In theory, it is well known that when the side length of the square aperture is the same as the diameter of the circular aperture, the coherence transfer function of the former is greater than that of the circular aperture. This means that the square aperture can collect more spatial frequency information in the spatial spectrum domain for each LR image. The experimental results show that the square aperture still has a good contrast of 0.4 in the case of groups (4,1) in the USAF target (Fig. 6). The high luminous flux of the square aperture is utilized to reduce the dynamic range of the data set, therefore reducing the image acquisition time. The small dynamic range recovery results are shown in Fig. 8, and the image still has good contrast.ConclusionsThis study demonstrates the macroscopic FP technique based on square aperture. The macroscopic FP technique has a better performance by replacing the conventional circular aperture scanning on the spatial spectrum plane with a square aperture. The numerical simulation and experimental verification show that when the square side length and circular diameter are the same, the square aperture has the advantages of high luminous flux and wide optical transfer function compared with the circular aperture. The wider spectrum of each LR image can increase the overlapping information between two adjacent images. The increase in luminous flux can reduce the dynamic range of image acquisition, thereby reducing the image acquisition time. As a result, the FP technique with the square aperture can achieve higher imaging resolution, speed, and signal-to-noise ratio. With the development of optical integration and micro/nano optical fabrication, the fabrication and integration of square aperture lens will be more convenient and can make full use of the aperture of the optical integrated devices. Considering the imaging sensor and its natural rectangular structure both in spatial and Fourier domains, the square aperture is more matched than the circular aperture. Square aperture lenses are promising to be integrated into various imaging systems and will appear in future miniaturized optical devices. ObjectiveFourier ptychography (FP) is a recently developed phase retrieval approach, which simultaneously realizes image restoration with super-resolution and a large field of view. This approach is also a novel super-resolution optical imaging technique that can be efficiently applied to microscopy and macroscopy respectively. However, one limitation of FP is the long acquisition and reconstruction time due to the numerous low-resolution (LR) images that are needed. Especially when the LR image is acquired under large angle illumination (dark field imaging), the low illumination intensity leads to the long exposure time of the imaging sensors. The situation becomes more time-consuming when a new sample is imaged, and thus a larger number of images with different exposure time for each LR image should be tested to obtain the corresponding best exposure time. Losing low gray value pixels exists in the FP algorithm preprocessing, which ultimately leads to the loss of image information due to the weak local image intensity in the recovery result. With the development of optical integration and micro/nano fabrication technology, the circular lens is no longer necessary. On the contrary, the square lens can fill the entire aperture of the miniaturized imaging system (cuboid) and collect more light fields. This paper introduces the square aperture lens into the FP approach which is conducive to collecting the complete aperture light field for further miniaturized imaging devices. The introduced square lens is expected to improve the image intensity and resolution of the FP approach, reduce the data acquisition time, and be more in line with the optical integration development.MethodsAccording to the Rayleigh criterion, the low resolution of the optical imaging system is caused by the lens aperture. The FP approach achieves super-resolution and large-field imaging by combining multiple LR images. Different from the FP microscopy, the macroscopic FP technique requires moving the imaging camera or using array imaging cameras to obtain a series of LR images of the object. Each LR image is obtained by locating the aperture on the spatial spectrum plane with different positions set in horizontal and vertical directions. After the LR image is acquired at the current position, the aperture is moved to the next position according to the required step size until the spatial spectrum is fully collected. To overcome the position offset of each LR image, the template matching method based on cross-correlation is applied for image registration. This paper studies the macroscopic FP imaging technology based on square aperture. Using the imaging sensor and its natural rectangular structure in the spatial Fourier domain is more effective to use the synthesis of square aperture to improve the imaging resolution and reduce the time consumption. The macroscopic FP technology based on circular aperture is also tested to make a better comparison. In the simulation, the ptychographical iterative engine (PIE) algorithm on the MATLAB platform is adopted to iteratively recover the phase to obtain a high-resolution image. In the experiment, a macroscopic FP optical setup is built. A square aperture, a lens, and a CCD are mounted on an optical bench as a comment imaging system. The system translated in the x-y direction to collect different spatial spectra and obtain a series of LR images. The PIE algorithm used in the simulation can recover the super-resolution image from the LR images.Results and DiscussionsWhen the circular aperture is used in the FP method, the image intensity is too weak as luminous flux lacks. The low gray value pixels would be missed during the FP algorithm preprocessing, which ultimately leads to image information loss due to the weak local image intensity in the recovery result. Compared to the circular aperture, the square aperture has a greater luminous flux and can obtain higher image intensity when collecting image information in the spatial spectrum plane. In theory, it is well known that when the side length of the square aperture is the same as the diameter of the circular aperture, the coherence transfer function of the former is greater than that of the circular aperture. This means that the square aperture can collect more spatial frequency information in the spatial spectrum domain for each LR image. The experimental results show that the square aperture still has a good contrast of 0.4 in the case of groups (4,1) in the USAF target (Fig. 6). The high luminous flux of the square aperture is utilized to reduce the dynamic range of the data set, therefore reducing the image acquisition time. The small dynamic range recovery results are shown in Fig. 8, and the image still has good contrast.ConclusionsThis study demonstrates the macroscopic FP technique based on square aperture. The macroscopic FP technique has a better performance by replacing the conventional circular aperture scanning on the spatial spectrum plane with a square aperture. The numerical simulation and experimental verification show that when the square side length and circular diameter are the same, the square aperture has the advantages of high luminous flux and wide optical transfer function compared with the circular aperture. The wider spectrum of each LR image can increase the overlapping information between two adjacent images. The increase in luminous flux can reduce the dynamic range of image acquisition, thereby reducing the image acquisition time. As a result, the FP technique with the square aperture can achieve higher imaging resolution, speed, and signal-to-noise ratio. With the development of optical integration and micro/nano optical fabrication, the fabrication and integration of square aperture lens will be more convenient and can make full use of the aperture of the optical integrated devices. Considering the imaging sensor and its natural rectangular structure both in spatial and Fourier domains, the square aperture is more matched than the circular aperture. Square aperture lenses are promising to be integrated into various imaging systems and will appear in future miniaturized optical devices.
Acta Optica Sinica
- Publication Date: Mar. 10, 2023
- Vol. 43, Issue 5, 0518001 (2023)
High-Speed Scanning of GHz Ultrafast Laser Using Acousto-Optic Deflection
Jing Zhang, Junpeng Wen, Zhe Zhu, Xiaoming Wei, and Zhongmin Yang
ObjectiveTwo-photon excitation microscopy is a powerful tool for studying brain neuronal activities. The imaging speed of traditional two-photon excitation microscopy technologies based on mechanical point-by-point laser scanning is relatively slow, which prevents the real-time observation of neuronal activities. Additionally, femtosecond lasers with high repetition rate are essential for high-speed two-photon excitation microscopy to achieve high signal intensity within a short pixel dwell time. We demonstrate a parallel GHz ultrafast laser scanning technology using acousto-optic deflection to exploit new potential for high-speed two-photon microscopy. The high-speed GHz ultrafast laser scanning system is built in the 920 nm wavelength range. By adjusting the temporal and spatial arrangement, 33 distinguishable parallel GHz ultrafast laser scanning beams are simultaneously generated within a frequency range of 15-31 MHz.MethodsWe adopt high-speed single-pixel parallel signal detection. The 920-nm femtosecond laser with a high repetition rate is split into two polarized beams using a polarizing beam splitter. One beam experiencing multitone-frequency modulation via an acousto-optic modulator serves as the reference beam, and the other beam is deflected by the radio frequency (RF) encoding technology. A time-domain signal with a random initial phase for each frequency drives the acousto-optic deflector to generate a one-dimensional laser beam array. The light spot is characterized by a CCD camera, then a delay line is employed to adjust the spatio-temporal overlap of the two beams to achieve interference. The electrical signals generated by the photodetector are digitally sampled by a high-speed data acquisition card and then are applied with a fast Fourier transform (FFT). Each laser beam is tagged with a specific frequency.Results and DiscussionsFrequency encoding design is performed within an RF range of 55-71 MHz to generate multitone RF driving signals with 33 frequencies, and each with a random initial phase. The duration of an arbitrary waveform cycle is set at 32.76 μs (Fig. 2). Although initially set with uniform amplitudes, the Fourier spectra of the loaded multi-frequency driving signals show variations after passing through the waveform generator and RF amplifier (Fig. 3). The acousto-optic deflector generates a one-dimensional laser beam array with relatively uniform intensities of the spots (Fig. 5), validating the correctness of the encoding scheme. After achieving spatio-temporal overlap, the photodetector detects typical beating signals with a duration of 32.76 μs (Fig. 8). A final RF spectrum of 33 uniformly spaced beating frequencies after performing FFT is obtained by multiple averaging (Fig. 10). The proposed parallel scanning technology presents promising applications in high-speed two-photon microscopy.ConclusionsWe design a high-speed parallel scanning system based on a 920 nm GHz ultrafast laser. The diffraction efficiency of the acousto-optic deflector is optimized, and a double-peak pattern is found within the driving frequency range of 30-90 MHz, with a 3 dB bandwidth of approximately 40.2 MHz. By designing an RF encoding scheme, the system generates 33 frequencies simultaneously. The generated diffraction laser beam array shows a nearly uniform intensity distribution. By adjusting the spatio-temporal overlap, each laser beam is frequency-tagged with a specific frequency. The RF spectrum of the beating signals after performing FFT is obtained by averaging the data multiple times to generate 33 distinguishable beating frequencies in the frequency range of 15-31 MHz. This confirms that the system can serve as a high-speed 0.9 μm two-photon laser parallel scanning light source. ObjectiveTwo-photon excitation microscopy is a powerful tool for studying brain neuronal activities. The imaging speed of traditional two-photon excitation microscopy technologies based on mechanical point-by-point laser scanning is relatively slow, which prevents the real-time observation of neuronal activities. Additionally, femtosecond lasers with high repetition rate are essential for high-speed two-photon excitation microscopy to achieve high signal intensity within a short pixel dwell time. We demonstrate a parallel GHz ultrafast laser scanning technology using acousto-optic deflection to exploit new potential for high-speed two-photon microscopy. The high-speed GHz ultrafast laser scanning system is built in the 920 nm wavelength range. By adjusting the temporal and spatial arrangement, 33 distinguishable parallel GHz ultrafast laser scanning beams are simultaneously generated within a frequency range of 15-31 MHz.MethodsWe adopt high-speed single-pixel parallel signal detection. The 920-nm femtosecond laser with a high repetition rate is split into two polarized beams using a polarizing beam splitter. One beam experiencing multitone-frequency modulation via an acousto-optic modulator serves as the reference beam, and the other beam is deflected by the radio frequency (RF) encoding technology. A time-domain signal with a random initial phase for each frequency drives the acousto-optic deflector to generate a one-dimensional laser beam array. The light spot is characterized by a CCD camera, then a delay line is employed to adjust the spatio-temporal overlap of the two beams to achieve interference. The electrical signals generated by the photodetector are digitally sampled by a high-speed data acquisition card and then are applied with a fast Fourier transform (FFT). Each laser beam is tagged with a specific frequency.Results and DiscussionsFrequency encoding design is performed within an RF range of 55-71 MHz to generate multitone RF driving signals with 33 frequencies, and each with a random initial phase. The duration of an arbitrary waveform cycle is set at 32.76 μs (Fig. 2). Although initially set with uniform amplitudes, the Fourier spectra of the loaded multi-frequency driving signals show variations after passing through the waveform generator and RF amplifier (Fig. 3). The acousto-optic deflector generates a one-dimensional laser beam array with relatively uniform intensities of the spots (Fig. 5), validating the correctness of the encoding scheme. After achieving spatio-temporal overlap, the photodetector detects typical beating signals with a duration of 32.76 μs (Fig. 8). A final RF spectrum of 33 uniformly spaced beating frequencies after performing FFT is obtained by multiple averaging (Fig. 10). The proposed parallel scanning technology presents promising applications in high-speed two-photon microscopy.ConclusionsWe design a high-speed parallel scanning system based on a 920 nm GHz ultrafast laser. The diffraction efficiency of the acousto-optic deflector is optimized, and a double-peak pattern is found within the driving frequency range of 30-90 MHz, with a 3 dB bandwidth of approximately 40.2 MHz. By designing an RF encoding scheme, the system generates 33 frequencies simultaneously. The generated diffraction laser beam array shows a nearly uniform intensity distribution. By adjusting the spatio-temporal overlap, each laser beam is frequency-tagged with a specific frequency. The RF spectrum of the beating signals after performing FFT is obtained by averaging the data multiple times to generate 33 distinguishable beating frequencies in the frequency range of 15-31 MHz. This confirms that the system can serve as a high-speed 0.9 μm two-photon laser parallel scanning light source.
Acta Optica Sinica
- Publication Date: Dec. 10, 2023
- Vol. 43, Issue 23, 2318001 (2023)
Retinal Vessel Segmentation via Self-Adaptive Compensation Network
Lin Zhang, Chuang Wu, Xinyu Fan, Chaoju Gong, Suyan Li, and Hui Liu
ObjectiveHuman eye is a crucial component of vision, but the number of patients suffering from ocular illnesses is growing every year. It has been discovered that the morphological characteristics of retinal blood vessels are strongly associated with several ocular conditions including diabetic retinopathy and glaucoma, and they are frequently employed in clinical diagnosis. Therefore, precise segmentation of retinal blood vessels based on color fundus images is crucial for the diagnosis of ocular illnesses. However, the fundus image itself displays noise, poor contrast, and an unbalanced distribution of blood vessels and background pixels. Additionally, morphological information gathering is challenging due to the delicate, highly curved, and multi-scale properties of retinal blood vessels. The time-consuming, difficult, and subjective nature of doctors' manual segmentation makes it ineffective for providing a large number of patients with a speedy diagnosis. To achieve precise automatic segmentation of retinal blood vessels from end to end, we propose the self-adaptive compensation network (SACom).MethodsSACom employs the U-shaped network as its fundamental structure. First, deformable convolution is incorporated into the encoder to enhance the model's capacity to learn information about morphological structures of retinal blood vessels. An adaptive multi-scale aligned context (AMAC) module is then developed at the bottom of the U-shaped network to extract and aggregate multi-scale context information and align the context features produced by pooling. It can adaptively extract context features according to the input image size and utilize the image context information correctly. Finally, a collaborative compensation branch (CCB) is proposed to fully leverage the feature layer in the decoder and high-level semantic features at the bottom of the network. Its multi-level outputs are helpful for positioning the overall structure of the blood vessel to fine details. Then they are fused with the output feature layer of the decoder end through the feature layer averaging adaptive fusion to improve the mapping capability of the model.Results and DiscussionsThe segmentation accuracy of retinal vessels can be effectively improved by the proposed SACom model. Each module is beneficial to improve segmentation performance according to the ablation experiment. Compared with the baseline model, SACom just adds a small number of extra parameters (Table 3). The proposed approach can thoroughly detect both thick blood vessels and thin blood vessels, and the connectedness of blood vessels is also more ideal, according to the visualization results of the segmentation (Fig. 6). Subsequent investigation reveals that there are microscopic blood vessels in the SACom segmentation results that are not labeled by experts but exist in fundus images (Fig. 7). It is clear that SACom has a good ability to segment blood vessels and identify blood vessel pixels more accurately, thereby addressing strong subjectivity in manual labeling. SACom performs better than other state-of-the-art methods generally (Table 5), with high sensitivity. The accuracy reaches 0.9695, 0.9763, and 0.9753, the sensitivities are 0.8403, 0.8748, and 0.8506, and the respective AUC values are 0.9880, 0.9917, and 0.9919 for DRIVE, CHASE_DB1, and STARE datasets, respectively.ConclusionsAn effective automatic segmentation algorithm called SACom is put forth to achieve precise segmentation of retinal vessels in fundus images. SACom integrates deformable convolution into the encoder based on the network architecture of U-Net to improve the learning capacity of vascular structural information. The bottom of the U-Net is constructed with an AMAC module that can collect and aggregate multi-scale aligned context information to adapt to the multi-scale issue of retinal blood vessels. Finally, a CCB is proposed. Its multi-level outputs calculate loss respectively and conduct backpropagation to improve the accuracy of each branch's output result. The outputs of the CCB are averaged and then adaptively fused with the output feature map of the decoder for accurate segmentation. The experimental results on three datasets reveal that the method has excellent generalization capability for different pixel classifications, especially for blood vessel pixels, and its comprehensive segmentation performance is better than other state-of-the-art algorithms. In addition, the proposed algorithm does not need too much computation load, which makes it easy to deploy in clinical applications. ObjectiveHuman eye is a crucial component of vision, but the number of patients suffering from ocular illnesses is growing every year. It has been discovered that the morphological characteristics of retinal blood vessels are strongly associated with several ocular conditions including diabetic retinopathy and glaucoma, and they are frequently employed in clinical diagnosis. Therefore, precise segmentation of retinal blood vessels based on color fundus images is crucial for the diagnosis of ocular illnesses. However, the fundus image itself displays noise, poor contrast, and an unbalanced distribution of blood vessels and background pixels. Additionally, morphological information gathering is challenging due to the delicate, highly curved, and multi-scale properties of retinal blood vessels. The time-consuming, difficult, and subjective nature of doctors' manual segmentation makes it ineffective for providing a large number of patients with a speedy diagnosis. To achieve precise automatic segmentation of retinal blood vessels from end to end, we propose the self-adaptive compensation network (SACom).MethodsSACom employs the U-shaped network as its fundamental structure. First, deformable convolution is incorporated into the encoder to enhance the model's capacity to learn information about morphological structures of retinal blood vessels. An adaptive multi-scale aligned context (AMAC) module is then developed at the bottom of the U-shaped network to extract and aggregate multi-scale context information and align the context features produced by pooling. It can adaptively extract context features according to the input image size and utilize the image context information correctly. Finally, a collaborative compensation branch (CCB) is proposed to fully leverage the feature layer in the decoder and high-level semantic features at the bottom of the network. Its multi-level outputs are helpful for positioning the overall structure of the blood vessel to fine details. Then they are fused with the output feature layer of the decoder end through the feature layer averaging adaptive fusion to improve the mapping capability of the model.Results and DiscussionsThe segmentation accuracy of retinal vessels can be effectively improved by the proposed SACom model. Each module is beneficial to improve segmentation performance according to the ablation experiment. Compared with the baseline model, SACom just adds a small number of extra parameters (Table 3). The proposed approach can thoroughly detect both thick blood vessels and thin blood vessels, and the connectedness of blood vessels is also more ideal, according to the visualization results of the segmentation (Fig. 6). Subsequent investigation reveals that there are microscopic blood vessels in the SACom segmentation results that are not labeled by experts but exist in fundus images (Fig. 7). It is clear that SACom has a good ability to segment blood vessels and identify blood vessel pixels more accurately, thereby addressing strong subjectivity in manual labeling. SACom performs better than other state-of-the-art methods generally (Table 5), with high sensitivity. The accuracy reaches 0.9695, 0.9763, and 0.9753, the sensitivities are 0.8403, 0.8748, and 0.8506, and the respective AUC values are 0.9880, 0.9917, and 0.9919 for DRIVE, CHASE_DB1, and STARE datasets, respectively.ConclusionsAn effective automatic segmentation algorithm called SACom is put forth to achieve precise segmentation of retinal vessels in fundus images. SACom integrates deformable convolution into the encoder based on the network architecture of U-Net to improve the learning capacity of vascular structural information. The bottom of the U-Net is constructed with an AMAC module that can collect and aggregate multi-scale aligned context information to adapt to the multi-scale issue of retinal blood vessels. Finally, a CCB is proposed. Its multi-level outputs calculate loss respectively and conduct backpropagation to improve the accuracy of each branch's output result. The outputs of the CCB are averaged and then adaptively fused with the output feature map of the decoder for accurate segmentation. The experimental results on three datasets reveal that the method has excellent generalization capability for different pixel classifications, especially for blood vessel pixels, and its comprehensive segmentation performance is better than other state-of-the-art algorithms. In addition, the proposed algorithm does not need too much computation load, which makes it easy to deploy in clinical applications.
Acta Optica Sinica
- Publication Date: Jul. 25, 2023
- Vol. 43, Issue 14, 1418001 (2023)
SHG Super-Resolution Microscopy Based on Point-Scanning Structured Light
Meiting Wang, Wenshuai Wu, Lei Wang, Xiaomin Zheng, Boliang Jia, Jiajie Chen, and Yonghong Shao
Second harmonic generation (SHG) imaging technology is a powerful tool capable of imaging non-centrosymmetric biological tissues without labeling, which has become an important means of life science research. Due to the diffraction limit, SHG technology cannot resolve the fine structures below the diffraction limit. Although super-resolution microscopy has made breakthrough progress, the coherent nonlinear process of SHG limits the development of the SHG super-resolution microscopy. A point-scanning structured illumination SHG super-resolution microscopy (SHG-psSIM) technology is proposed to realize the super-resolution SHG microscopic imaging of zinc oxide particles and mouse tail tendons. The electro-optic modulator is introduced into the excitation light path of the traditional SHG microscopy system. A point-scanning structured illumination pattern is generated by sinusoidally modulating of the excitation beam. Based on the moiré fringe effect generated by interaction of the point-scanning structured illumination pattern with the sample structures, the undetectable high frequency information of samples is moved to the observable passband of the microscopy and detected by the photomultiplier. Finally, the algorithm software is used to reconstruct the super-resolution image. Compared with the traditional SHG system, the resolution of SHG-psSIM is improved by 1.86 times. Second harmonic generation (SHG) imaging technology is a powerful tool capable of imaging non-centrosymmetric biological tissues without labeling, which has become an important means of life science research. Due to the diffraction limit, SHG technology cannot resolve the fine structures below the diffraction limit. Although super-resolution microscopy has made breakthrough progress, the coherent nonlinear process of SHG limits the development of the SHG super-resolution microscopy. A point-scanning structured illumination SHG super-resolution microscopy (SHG-psSIM) technology is proposed to realize the super-resolution SHG microscopic imaging of zinc oxide particles and mouse tail tendons. The electro-optic modulator is introduced into the excitation light path of the traditional SHG microscopy system. A point-scanning structured illumination pattern is generated by sinusoidally modulating of the excitation beam. Based on the moiré fringe effect generated by interaction of the point-scanning structured illumination pattern with the sample structures, the undetectable high frequency information of samples is moved to the observable passband of the microscopy and detected by the photomultiplier. Finally, the algorithm software is used to reconstruct the super-resolution image. Compared with the traditional SHG system, the resolution of SHG-psSIM is improved by 1.86 times.
Acta Optica Sinica
- Publication Date: May. 08, 2022
- Vol. 42, Issue 10, 1018001 (2022)
Research on Microscopic Imaging of High Resolution Light Field Based on Graph Regularization
Feifan Yang, Hui Li, Jing Peng, and Yuntao Wu
With regard to the low spatial resolution in light filed microscopy, we proposed a method based on graph regularization and reconstructed a super-resolution light field. As a result, high-resolution view angle images of the light field were obtained. Firstly, the images of the light field were aperiodically extracted. Then, the super-resolution issue in the light field was transformed into a global optimization problem, in which the complementary information between those images was applied for smoothing regularization. Finally, the gradient descent algorithm minimized the objective function and thus reconstructed the high-resolution view angle images. Experimentally, a light field microscope, equipped with an objective lens, a microlens array, and a CCD, collected the data of the light field, and then graph regularization was used to reconstruct the high-resolution light field. Compared with the traditional methods, the proposed method has small calculation amount in disparity estimation and high imaging quality, and effectively retains the original light field structure. With regard to the low spatial resolution in light filed microscopy, we proposed a method based on graph regularization and reconstructed a super-resolution light field. As a result, high-resolution view angle images of the light field were obtained. Firstly, the images of the light field were aperiodically extracted. Then, the super-resolution issue in the light field was transformed into a global optimization problem, in which the complementary information between those images was applied for smoothing regularization. Finally, the gradient descent algorithm minimized the objective function and thus reconstructed the high-resolution view angle images. Experimentally, a light field microscope, equipped with an objective lens, a microlens array, and a CCD, collected the data of the light field, and then graph regularization was used to reconstruct the high-resolution light field. Compared with the traditional methods, the proposed method has small calculation amount in disparity estimation and high imaging quality, and effectively retains the original light field structure.
Acta Optica Sinica
- Publication Date: May. 08, 2021
- Vol. 41, Issue 9, 0918001 (2021)
Light Sheet Fluorescence Microscope Based on Line-Scanning Imaging
Yueqian Wu, Xin Xu, Jixiang Wang, Xin Zhang, and Guohua Shi
An updated light sheet fluorescence microscope (LSFM), combining the digital scanning light sheet fluorescence microscope (DSLM) and the line-scanning imaging (LSI) method, is designed to improve the imaging quality by decreasing sample scattering. Based on DSLM, a scanning galvanometer for descanning is added in the detection light path. Moreover, the scanning galvanometer in the illumination light path is synchronized with the descanning galvanometer to fix the evenly moving images at the same position in front of the camera during the control process, thereby realizing LSI. The linear array detector is simulated by the area array camera in the system for conveniently comparing with the traditional method. In addition, compared with the conventional LSFM, the improved system is more effective in suppressing the sample scattering problem in the imaging experiments of high-scattering fluorescent microsphere samples and zebrafish heart samples. Hence, the feasibility of LSFM based on LSI method can be verified. An updated light sheet fluorescence microscope (LSFM), combining the digital scanning light sheet fluorescence microscope (DSLM) and the line-scanning imaging (LSI) method, is designed to improve the imaging quality by decreasing sample scattering. Based on DSLM, a scanning galvanometer for descanning is added in the detection light path. Moreover, the scanning galvanometer in the illumination light path is synchronized with the descanning galvanometer to fix the evenly moving images at the same position in front of the camera during the control process, thereby realizing LSI. The linear array detector is simulated by the area array camera in the system for conveniently comparing with the traditional method. In addition, compared with the conventional LSFM, the improved system is more effective in suppressing the sample scattering problem in the imaging experiments of high-scattering fluorescent microsphere samples and zebrafish heart samples. Hence, the feasibility of LSFM based on LSI method can be verified.
Acta Optica Sinica
- Publication Date: Oct. 29, 2021
- Vol. 41, Issue 20, 2018001 (2021)
Wide-Field Error Correction Method for Parallel Differential Confocal Axial Measurement
Yiqing Ye, Dingrong Yi, Wei Jiang, Linghua Kong, and Caihong Huang
The parallel differential confocal three-dimensional imaging method realizes multi-point parallel detection by dividing the illumination light into multiple isolated beams, which solves the problem of low imaging efficiency caused by point-by-point scanning of traditional differential confocal imaging. It is an ideal fast and high-precision three-dimensional imaging method. However, both non-homogeneous illumination and field curvature of an optical imaging system induce errors in the measurement results during the process of parallel differential confocal detection. The effects of inevitable non-homogeneous illumination and field curvature of an optical system during the wide-field imaging process on the parallel differential confocal axial measurement are analyzed, a wide-field error correction method is proposed, and a theoretical model of wide-field error correction is established for the parallel differential confocal axial measurement. The experimental results show that the proposed method can effectively suppress the influence of non-homogeneous illumination on the measurement results and correct the axial measurement error caused by the field curvature of the optical imaging system. It ensures the universality of parallel differential confocal nanometer-scale axial measurement accuracy in the full field of view. The implementation process of this method is simple and convenient, and it is also applicable to error correction of other parallel confocal detection methods. The parallel differential confocal three-dimensional imaging method realizes multi-point parallel detection by dividing the illumination light into multiple isolated beams, which solves the problem of low imaging efficiency caused by point-by-point scanning of traditional differential confocal imaging. It is an ideal fast and high-precision three-dimensional imaging method. However, both non-homogeneous illumination and field curvature of an optical imaging system induce errors in the measurement results during the process of parallel differential confocal detection. The effects of inevitable non-homogeneous illumination and field curvature of an optical system during the wide-field imaging process on the parallel differential confocal axial measurement are analyzed, a wide-field error correction method is proposed, and a theoretical model of wide-field error correction is established for the parallel differential confocal axial measurement. The experimental results show that the proposed method can effectively suppress the influence of non-homogeneous illumination on the measurement results and correct the axial measurement error caused by the field curvature of the optical imaging system. It ensures the universality of parallel differential confocal nanometer-scale axial measurement accuracy in the full field of view. The implementation process of this method is simple and convenient, and it is also applicable to error correction of other parallel confocal detection methods.
Acta Optica Sinica
- Publication Date: Sep. 18, 2020
- Vol. 40, Issue 20, 2018001 (2020)
Microscope Autofocus Research Based on Stripe Projection
Lu Liu, Peizheng Yan, Xizuo Dan, Siyuan Bao, and Yonghong Wang
Autofocus technology is a crucial part in the detection of thin film transistor liquid crystal display (TFT-LCD). Because the fast and accurate autofocus is required, an autofocus microscope system is adopted to detect TFT-LCD. The digital grating is projected onto the surface of the measured object. The reflected light is split into two paths using a beam-splitting prism. One is imaged onto the area-array charge-coupled device (CCD) directly, whereas the other is split into two paths using another beam-splitting prism again. A reflecting mirror is used to couple two images of the digital grating onto a line-array CCD. The defocusing direction and defocusing amount are judged by comparing the sharpness of two grating images on the line-array CCD. In addition, a micro-displacement platform is used to move the objective lens and achieve the microscope autofocus. Thus, this study analyzes the digital grating, which affects autofocus, and uses a variable-period digital grating for focusing. Autofocus technology is a crucial part in the detection of thin film transistor liquid crystal display (TFT-LCD). Because the fast and accurate autofocus is required, an autofocus microscope system is adopted to detect TFT-LCD. The digital grating is projected onto the surface of the measured object. The reflected light is split into two paths using a beam-splitting prism. One is imaged onto the area-array charge-coupled device (CCD) directly, whereas the other is split into two paths using another beam-splitting prism again. A reflecting mirror is used to couple two images of the digital grating onto a line-array CCD. The defocusing direction and defocusing amount are judged by comparing the sharpness of two grating images on the line-array CCD. In addition, a micro-displacement platform is used to move the objective lens and achieve the microscope autofocus. Thus, this study analyzes the digital grating, which affects autofocus, and uses a variable-period digital grating for focusing.
Acta Optica Sinica
- Publication Date: Aug. 05, 2019
- Vol. 39, Issue 8, 0818001 (2019)
Improvement of Imaging Resolution of Fluorescence Emission Difference Super-Resolution Microscopy with Annular Pupil
Suhui Deng, Jie Hu, Yiping Xiao, and Mingping Liu
Super-resolution imaging can be achieved by fluorescence emission difference (FED) microscopy through the subtraction of a negative confocal image scanned by a hollow focal spot from a confocal image scanned by a solid focal spot. This study proposes an easily accessible annular pupil filter to enhance the resolution of deformation-free FED imaging. The hollow spot size is reduced by an appropriate annular pupil filter in the light path of hollow spot. Meanwhile, a diaphragm with a high numerical aperture is permitted for the beam of a solid focal spot, and a small solid focal spot which is matching well with the hollow focal spot is achieved. The spatial resolution of FED microscopy is increased under the effects of these two aspects. Super-resolution imaging can be achieved by fluorescence emission difference (FED) microscopy through the subtraction of a negative confocal image scanned by a hollow focal spot from a confocal image scanned by a solid focal spot. This study proposes an easily accessible annular pupil filter to enhance the resolution of deformation-free FED imaging. The hollow spot size is reduced by an appropriate annular pupil filter in the light path of hollow spot. Meanwhile, a diaphragm with a high numerical aperture is permitted for the beam of a solid focal spot, and a small solid focal spot which is matching well with the hollow focal spot is achieved. The spatial resolution of FED microscopy is increased under the effects of these two aspects.
Acta Optica Sinica
- Publication Date: Jul. 15, 2019
- Vol. 39, Issue 7, 0718001 (2019)
Topics
© Copyright 2018-2021 | Chinese Laser Press. All Rights Reserved 沪ICP备15018463号-20