Two-photon microscopy (TPM) imaging has been widely used in many fields, such as in vivo tumor imaging, neuroimaging, and brain disease research. However, the small field-of-view (FOV) in two-photon imaging (typically within diameter of 1 mm) limits its further application. Although the FOV can be extended through adaptive optics technology, the complex optical paths, additional device costs, and cumbersome operating procedures limit its promotion. In this study, we propose the use of deep learning technology instead of adaptive optics technology to expand the FOV of two-photon imaging. The large FOV of TPM can be realized without additional hardware (such as a special objective lens or phase compensation device). In addition, a BN-free attention activation residual U-Net (nBRAnet) network framework is designed for this imaging method, which can efficiently correct aberrations without requiring wavefront detection.
Commercially available objectives have a nominal imaging FOV that has been calibrated by the manufacturer. Within the nominal FOV, the objective lens exhibits negligible aberrations. However, the aberrations increase dramatically beyond the nominal FOV. Therefore, the imaging FOV of the objective lens is limited to its nominal FOV. In this study, we improved the imaging quality of the FOV outside the nominal region by combining adaptive optics (AO) and deep learning. Aberrant and AO-corrected images were collected outside the nominal FOV. Thus, we obtained a paired dataset consisting of AO-corrected and uncorrected images. A supervised neural network was trained using the aberrated images as the input and the AO-corrected images as the output. After training, the images collected from regions outside the nominal FOV could be fed directly to the network. Aberration-corrected images were produced, and the imaging system could be used without AO hardware.
The experimental test results include the imaging results of samples such as fluorescent beads with diameter of 1 μm and Thy1-GFP and CX3CR1-GFP mouse brain slices, and the results of the corresponding network output. The high peak signal-to-noise ratio (PSNR) values of the test output and ground truth demonstrate the feasibility of extending the FOV to TPM imaging using deep learning. At the same time, the intensity contrast between the nBRAnet network output image and the ground truth on the horizontal line is compared in detail (Figs. 3, 4, and 5). The extended FOV of different samples is randomly selected for analysis, and a high degree of coincidence is observed in the intensity comparison. The experimental results show that after using the network, both the resolution and fluorescence intensity can be restored to a level where there is almost no aberration, which is close to the result after correcting using AO hardware. To demonstrate the advantages of the network framework designed in this study, the traditional U-Net structure and the very deep super-resolution (VDSR) model are used to compare with ours. When using the same training dataset to train different models, we find that the experimental results of the VDSR model contain a considerable amount of noise, whereas the experimental results of the U-Net network lose some details (Fig. 6). The high PSNR values clearly demonstrate the strength of our nBRAnet network framework (Table 3).
This study provides a novel method to effectively extend the FOV of TPM imaging by designing an nBRAnet network framework. In other words, deep learning was used to enhance the acquired image and expand the nominal FOV for commercial objectives. The experimental results show that images from extended FOVs can be restored to their AO-corrected versions using the trained network. That is, deep learning technology could be used instead of AO hardware technology to expand the FOV of commercially available objectives. This simplifies the operation and reduces the system cost. An extended FOV obtained using deep learning can be employed for cross-regional or whole-brain imaging.
