Three-dimensional (3D) imaging is an important research direction in microscopy and has been applied to many fields such as biomedicine and engineering science. Typical 3D microscopy techniques, such as laser confocal microscopy and multi-photon microscopy, are based on laser point scanning geometry, and the imaging speed is limited by the scanning speed; therefore, biological samples are likely to be damaged under long scanning durations and high-intensity illumination.
Recently, wide-field microscopy with 3D imaging capability has received significant attention. Wide-field microscopy can yield complete two-dimensional imaging simultaneously and affords temporal resolutions higher than spot scanning by two to three orders of magnitude. Additionally, wide-field imaging offers high-quality grayscale images and fewer samples to be damaged, thus rendering it suitable for the real-time observation of living samples. However, conventional wide-field microscopy suffers from defocused backgrounds and low axial resolutions. Owing to the rapid development of computer science and optical technology, various algorithms and techniques for processing wide-field images have been proposed to improve their axial resolutions, thus providing more possibilities for 3D imaging.
We focus on three types of rapid 3D wide-field microscopy techniques, i.e., shape of focus (SFF), structural illumination microscopy (SIM), and deep learning-assisted 3D imaging. The SFF technique enables the extraction of focal plane information by processing a series of image stacks and reconstructing the 3D morphology of samples without requiring specific hardware. In SIM, samples are illuminated by phase-shifted light fields with high spatial frequencies, images are captured using a CCD camera, and the in- and out-focus information can be effectively separated using decoding algorithms. Deep learning models can learn the mapping relationship between different types of images from a large amount of data, such as the conversion between wide-field images and confocal images; this is a simple method to obtain high-quality images. The trained model can remove the background information of wide-field microscopic images to improve the axial resolution of imaging, thus facilitating the realization of 3D imaging via wide-field microscopy.
It is believed that rapid 3D microscopes based on wide-field imaging will be applied to many fields such as biomedicine, materials science, and precision manufacturing in the near future.
This paper focuses on three rapid wide-field 3D imaging techniques, namely, SFF, SIM, and deep learning-assisted 3D imaging.
In the SFF technique, a focusing evaluation operator is used to calculate and extract the highest focus position of each pixel from a wide-field image stack; subsequently, the 3D depth image of the sample is reconstructed via a recovery algorithm, which is mainly used for surface topography measurement. We investigate the effect of the focused evaluation operator on the calculation results yielded by the SFF technique in different cases. Additionally, we discuss the development and application of the focused topography recovery operator and the optimization of related hardware.
Optical sectioning SIM utilizes encoded structured light fields to illuminate the sample and then recovers the 3D information of the sample using decoding algorithms, which can be used for both fluorescent and non-fluorescent imaging. We introduce the theoretical basis of optical sectioning SIM and then propose various rapid decoding algorithms for improving the reconstruction speed. Then, we discuss the development of related techniques and their most recent applications in the field of 3D color imaging.
Deep learning-assisted 3D imaging applies the learning ability of neural network models to complete target image tasks, such as the conversion between wide-field and confocal images as well as that between wide-field microscopy and SIM so as to achieve wide-field 3D imaging. We present the theoretical basis of deep learning-related models. Subsequently, we discuss the development and application of deep learning models for conversion between wide-field and confocal images as well as that between wide-field microscopy and SIM, followed by the applications of deep learning for achieving more rapid SIM imaging.
Finally, we discuss the current problems and future research directions for rapid 3D wide-field microscopy techniques.
Rapid 3D wide-field microscopy techniques have demonstrated performance improvement either through hardware modification or software assistance. However, these techniques are not perfect. SFF combined with other techniques is expected to benefit deep tissue imaging. The amount of SIM imaging data is two to three times that of the conventional wide-field microscopy, and the imaging speed of SIM can be further improved. Deep learning can be flexibly combined with other technologies. In summary, the potential of wide-field microscopy with 3D imaging capability is yet to be realized. Progress in technology and cross integration will enable the routine use of rapid 3D wide-field microscopy techniques in biomedical laboratories.
