• Photonics Research
  • Vol. 10, Issue 5, 1255 (2022)
Jiazhen Zhai1、†, Ruheng Shi1、†, and Lingjie Kong1、2、*
Author Affiliations
  • 1State Key Laboratory of Precision Measurement Technology and Instruments, Department of Precision Instrument, Tsinghua University, Beijing 100084, China
  • 2IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing 100084, China
  • show less
    DOI: 10.1364/PRJ.451895 Cite this Article Set citation alerts
    Jiazhen Zhai, Ruheng Shi, Lingjie Kong. Improving signal-to-background ratio by orders of magnitude in high-speed volumetric imaging in vivo by robust Fourier light field microscopy[J]. Photonics Research, 2022, 10(5): 1255 Copy Citation Text show less

    Abstract

    Fourier light field microscopy (FLFM) shows great potential in high-speed volumetric imaging of biodynamics. However, due to the inherent disadvantage of wide-field illumination, it suffers from intense background, arising from out of the depth-of-field signal and tissue scattered noise. The background will not only deteriorate the image contrast, making quantitative measurement difficult, but also introduce artifacts, especially in functional imaging of the neuronal network activity in vivo. Here, we propose the robust Fourier light field microscopy (RFLFM), which suppresses the background in FLFM by introducing structured illumination and computational reconstruction based on HiLo. The superior performance of RFLFM is verified by volumetric imaging of biological dynamics in larval zebrafish and mouse in vivo, at a volumetric imaging rate up to 33.3 Hz. The statistical results show that the fluorescence background can be significantly depressed, with the signal-to-background ratio improved by orders of magnitude and the whole image contrast improved by as much as 10.4 times. Moreover, we stress that, in functional imaging of neuronal network activity in turbid brain tissues, our system can avoid artifacts resulting from background fluctuations, while conventional light field microscopy fails. As a simple but powerful tool, we anticipate our technique to be widely adopted in robust, high-contrast, high-speed volumetric imaging.

    1. INTRODUCTION

    Benefiting from the capability of high-speed volumetric imaging, light field microscopy (LFM) [1] has found various applications in biomedical study in vivo, including fast functional imaging [2,3] and structural imaging [4,5]. However, the cost of volumetric imaging under a single exposure in LFM is low spatial resolution. LFM also is susceptible to reconstruction artifacts [68].

    Instead, Fourier light field microscopy (FLFM) is proposed [911], in which a lens is added to transform the native image to the Fourier domain on its back focal plane and the microlens array (MLA) is placed there to collect multi-view images. Compared to LFM, FLFM has the advantage of a uniform point spread function (PSF) across the field of view (FOV), which can remove reconstruction artifacts during deconvolution, achieving high optical resolution and high localization precision [1214]. In addition, the multi-angle projection method is also proposed to achieve full NA volumetric imaging, which is similar to FLFM, but has a lower imaging rate [15]. However, the issue of an intense background still exists in FLFM, which is also an issue in LFM. Considering that it inherits the conventional wide-field illumination strategy, the background in FLFM images is mainly caused by the excited signals from out of depth-of-field (DOF) and the tissue scattered emission light [16,17], which leads to poor contrast and unexcepted artifacts in practical imaging.

    To minimize the background for quantitative imaging in either LFM or FLFM, the computational LFM method [18], speckle illumination [19], light sheet illumination [2023], two-photon illumination [24], and confocal detection [25] are employed and all achieved excellent performances. However, these methods either require prior assumptions, restrict the sample size, or are not flexible for various applications. Moreover, selective-volume illumination methods [15] such as the light sheet illumination method fail in resisting the background from tissue scattering.

    Here, we propose what we believe, to the best of our knowledge, is a novel approach called robust Fourier light field microscopy (RFLFM) to achieve robust, high-speed, high-contrast volumetric imaging by removing the background signal based on structured illumination [2631] and computational reconstruction. Different from conventional FLFM, we use structured illumination and uniform illumination to sequentially modulate the designed DOF region. Then, the captured raw images are segmented and processed with HiLo algorithm [29] to subtract background information, which can improve the image contrast significantly and help to avoid artifacts induced by background fluctuations. After that, a high-quality and robust volumetric image can be reconstructed based on deconvolution as in conventional FLFM [25]. To demonstrate the superior performance of our technique, we perform both structural imaging and functional imaging in larval zebrafish and mouse in vivo, at volumetric imaging rate up to 33.3 Hz. Based on the statistic results, the signal-to-background ratio (SBR) can be improved by orders of magnitude, while the whole image contrast is improved by as much as 10.4 times. Especially in the functional imaging of neuronal network activity in turbid brain tissues in vivo, RFLFM generates robust and clean fluorescence dynamics, while conventional FLFM fails. Our system also maintains high resolution, large FOV, and large DOF, which makes it promising for robust, high-speed, high-contrast volumetric imaging of biodynamics in vivo.

    2. SYSTEM DESIGN

    The main principle of RFLFM is employing structure illumination and computational reconstruction based on HiLo algorithm in FLFM to achieve background-robust imaging, as described in Supplementary Note 1 in Ref. [32].

    Figure 1 shows the schematic layout of the RFLFM system. We use a collimated LED light (120LED X-Cite) as the light source, and an excitation bandpass filter (EF1, MF469-35, Thorlabs) to select λ=470  nm light for illumination. A digital micromirror device (DMD) (1080×1920  pixels, pixel size 10.8 μm, DLP9500, Texas Instruments) is adopted to switch between structured illumination and uniform illumination. To make the system compact, we use a total internal reflection (TIR) prism to separate the incident beam and the reflected beam on the DMD [33]. The patterns on DMD are further relayed to the native object plane (NOP) with two 4f relay systems, including relay system 1 (not shown in Fig. 1, composed of an AC508-150-A and an AC508-300-A, Thorlabs), and relay system 2, composed of relay lens (AC508-200-A, Thorlabs) and the objective [25× magnification, NA 1.05, XLPLN25XWMP2, Olympus], followed by the dichroic mirror (DMLP490, Thorlabs). For the structured illumination situation, a defined depth region is modulated with the predesigned period pattern, and the pattern contrast decreases quickly outside the designed region [34], as shown in Fig. S1 of Ref. [32]. The sample is placed on the xyz stage (MT3/M-Z8, Thorlabs) and the emitted fluorescence signals are collected by the objective and then reflected by the dichroic mirror. For the selected objective, the back-pupil diameter is 15.12 mm. We use a standard f=200  mm lens (AC508-200-A, Thorlabs) as a tube lens to acquire an image of 27.78× magnification with 1.05 NA at the native image plane (NIP), where we place a customized emission filter (EF2, Φ=30  mm, λ=525  nm, bandwidth 50 nm, Edmund). Then we choose an f=300  mm lens (AC508-300-A, Thorlabs) as the Fourier lens (FL); therefore, we can get the spatial frequency spectrum of signals with a 22.68 mm optical aperture diameter on the back focal plane of the FL. To get a large FOV with a high spatial resolution and large DOF in multi-views, we use a microlens array (MLA, FEL-46S03-38.24PM, Sigma, 3×4  mm pitch, f=38.24  mm) to segment the frequency spectrum, and finally get subimages of different views (31 views in our system, as shown in Fig. S2 in Ref. [32]) at a 3.54× magnification and a 0.1389 NA. A CMOS camera (5120×5120  pixels, pixel size 4.5  μm, S-25A80 CoaXPress, Adimec) is adopted to record the image, which is adjusted carefully to match the focal plane of the MLA. We synchronize the camera and DMD by a microcontroller (UNO Rev3, Arduino).

    System scheme of RFLFM. EF1, excitation filter 1; TIR, total internal reflection prism; DMD, deformable mirror device; RL, relay lens; DM, dichroic mirror; RM, reflector mirror; TL, tube lens; EF2, emission filter 2; FL, Fourier lens; MLA, microlens array. A DMD is used in the illumination path to project the uniform and structured illumination patterns, and a TIR is used to separate the incident beam and reflected beam on the DMD. The camera exposure is synchronized with each illumination pattern by the computer. A conventional FLFM imaging path is built to record images at different views. The inset shows the distribution of spatial frequency domain on the MLA.

    Figure 1.System scheme of RFLFM. EF1, excitation filter 1; TIR, total internal reflection prism; DMD, deformable mirror device; RL, relay lens; DM, dichroic mirror; RM, reflector mirror; TL, tube lens; EF2, emission filter 2; FL, Fourier lens; MLA, microlens array. A DMD is used in the illumination path to project the uniform and structured illumination patterns, and a TIR is used to separate the incident beam and reflected beam on the DMD. The camera exposure is synchronized with each illumination pattern by the computer. A conventional FLFM imaging path is built to record images at different views. The inset shows the distribution of spatial frequency domain on the MLA.

    Based on the optical design above, the DMD is demagnified 13.89 times to 1493  μm×840  μm when projected on the NOP. Thus, to ensure that the DMD modulated illumination could cover the whole FOV, we add a diaphragm at the NIP to restrict the FOV to Ø=840  μm. We divide the sensor plane to 666×666 pixels for each microlens, and get 31 effective subimages of different views, each of Ø with 666 effective pixels, as shown in Fig. S2 in Ref. [32]. According to Rayleigh criterion, the lateral optical resolution is 1.87 μm (λex=520  nm, NA=0.1389). However, since the system is working at 3.54× magnification, to ensure the Nyquist sampling ratio, the laterally optical resolution is limited to 2.54 μm. Thus, our expected DOF is about 85 μm at FWHM intensity (introduced in Supplementary Note 1 in Ref. [32]). Based on this, we choose the optical sectioning capability (i.e., FWHM, in the axial direction) of HiLo algorithm as 90 μm, which can be calculated by FWHMaxial=0.54Ks·NAillumination,where Ks=12Δs, and Δs represents the grid circle [31]. Limited by an aperture in the optical path, the illumination NA is about 0.92 here, so the grid circle should be 77 μm. In practice, to make sure the modulated region could cover every DOF (the machining error of MLA may cause focusing errors), we enlarge the grid circle by 20 percent; i.e., the grid circle is chosen as 93 μm on the NOP, corresponding to 1.29 mm on the DMD, as shown in Fig. S1 in Ref. [32]. The post-process methods (as shown in Fig. S3 in Ref. [32]) are introduced in Appendixes A and B.

    3. EXPERIMENTAL RESULTS

    All procedures involving animals are approved by the Animal Care and Use Committees of Tsinghua University.

    A. High-Contrast Volumetric Imaging of Vascular Structure in the Brains of Larval Zebrafish in vivo

    Imaging of vascular morphology and structure in larval zebrafish is a very attractive subject of clinical interests [35]. To demonstrate our advantage in high-contrast volumetric imaging, we perform imaging of the vascular structure in the brains of larval zebrafish in vivo. We use Tg (fli1a:GFP) zebrafish at 5–7 days’ post fertilization, and embed them in 1% agarose during imaging.

    We reconstruct a volume of 840  μm×840  μm×90  μm in both RFLFM and FLFM modes, as shown in Figs. S4(a) and S4(b) in Ref. [32], respectively. Note that the latter one is based on extracting the raw images under uniform illumination only. The same procedure will be performed below with no specific clarification. It suggests that our RFLFM could distinguish the vascular structure clearly while reducing the background fluorescence. The contrast (defined in Supplementary Note 1 in Ref. [32]) of the total image improves from 0.429 to 0.964 in RFLFM, with a 2.2 times improvement. To visually compare the remaining background intensity between the two methods, we show the profiles of the selected vascular structure in Fig. S4 in Ref. [32], which suggests that the SBR of the selected vascular structure is improved by as high as 16.8 times.

    B. High-Contrast Volumetric Imaging of Neuronal Network Activity in the Brains of Larval Zebrafish in vivo

    LFM is good for high-speed volumetric imaging of biodynamics. To demonstrate the advantage of our RFLFM in high-contrast volumetric imaging at a sub-cellular resolution, we perform calcium imaging of neural network activity in the brains of larval zebrafish in vivo. We use Tg (HUC:H2B-GCaMP6f) zebrafish at 5–7 days’ post fertilization, and embed them in 1% agarose during imaging. We perform volumetric imaging over a 90 μm depth range, and set the acquisition time for each volume as 100 ms (exposure time: 90 ms), corresponding to a volume rate of 5 Hz in RFLFM mode.

    To demonstrate the superiority of RFLFM in depressing the background signals, we compare the performance of RFLFM and conventional FLFM. We reconstruct a volume of 650  μm×650  μm×90  μm over 400 frames (80 s), as shown in Fig. 2. Single neurons can be identified in both RFLFM [Figs. 2(a) and 2(d)] and FLFM [Figs. 2(b) and 2(e)], which suggest subcellular resolution of our system. However, the conventional FLFM suffers from intense background fluorescence [Figs. 2(b), 2(f), 2(g), 2(j), and 2(l)], whereas RFLFM achieves volumetric images with much higher contrast [Figs. 2(a), 2(d), 2(e), 2(i), and 2(k)], and the image contrast is improved from 0.598 to 1.350 in RFLFM. In Fig. 2(c), we show the neural activity of 1141 neurons at seven planes (depths: 45  μm, 30  μm, 15  μm, 0, 15 μm, 30 μm, and 45 μm, the activities of individual neurons at each plane can be found in Fig. S5 in Ref. [32]) in the volume, which is extracted from RFLFM reconstructed images by the non-negative matrix factorization method [36]. In Fig. 2(h), we show a zoom-in view of 60 neurons’ fluorescence dynamics as examples.

    High-contrast volumetric imaging of neural network activity in the brains of larval zebrafish in vivo. The imaging depth is centered at about tens of microns below the brain surface of the zebrafish. (a) and (b) Maximum intensity projects (MIPs) over recording time in the 4D (x–y–z–t) domain, captured by RFLFM and FLFM modes, respectively. Scale bar: 150 μm. (c) Fluorescence signals of neurons, based on RFLFM reconstructed images. The activity shows a visible increase at about 57 s and 75 s. (d) 3D projection of (a) in different orthogonal planes. Scale bar: 60 μm. (g) 3D projection of (b) in different orthogonal planes. Scale bar: 60 μm. (e) and (f) Zoom-in views of the regions in boxes of (d) and (g), respectively. Scale bar: 20 μm. (h) Zoom-in view of orange box indicating the activity of 60 neurons in (c). (i)–(l) Zoom-in views of the regions in boxes of (e) and (f), where the arrows indicate the neurons. Scale bar: 10 μm. (m) Calcium tracings of five neurons, indicated in (i)–(l). The orange line indicates signals achieved in RFLFM and the blue line indicates signals enlarged three times achieved in FLFM.

    Figure 2.High-contrast volumetric imaging of neural network activity in the brains of larval zebrafish in vivo. The imaging depth is centered at about tens of microns below the brain surface of the zebrafish. (a) and (b) Maximum intensity projects (MIPs) over recording time in the 4D (xyzt) domain, captured by RFLFM and FLFM modes, respectively. Scale bar: 150 μm. (c) Fluorescence signals of neurons, based on RFLFM reconstructed images. The activity shows a visible increase at about 57 s and 75 s. (d) 3D projection of (a) in different orthogonal planes. Scale bar: 60 μm. (g) 3D projection of (b) in different orthogonal planes. Scale bar: 60 μm. (e) and (f) Zoom-in views of the regions in boxes of (d) and (g), respectively. Scale bar: 20 μm. (h) Zoom-in view of orange box indicating the activity of 60 neurons in (c). (i)–(l) Zoom-in views of the regions in boxes of (e) and (f), where the arrows indicate the neurons. Scale bar: 10 μm. (m) Calcium tracings of five neurons, indicated in (i)–(l). The orange line indicates signals achieved in RFLFM and the blue line indicates signals enlarged three times achieved in FLFM.

    To quantitatively show the optical-sectioning capability and its robustness to scattering-induced cross-talk, we compare the fluorescence dynamics (ΔF/F) of GCaMP measured from selected regions-of-interest (ROIs) in the RFLFM and FLFM [Figs. 2(i)–2(l)]. Considering the weak ΔF/F resulting from the intense background in the FLFM, we multiply its ΔF/F signals by a factor of 3, as shown in Fig. 2(m). It can be seen that the RFLFM shows signals of much higher SBR, thanks to the suppression of background fluorescence. Furthermore, we investigate the correlations of neural activity between pairs of neurons to study whether the ΔF/F detected from each neuron reflects its own activity or is overwhelmed by the background [37]. The results show that both the average and median correlation coefficients are <0.2, which indicates that the calcium traces detected from each neuron are not overwhelmed by the background fluorescence (Appendix C).

    Furthermore, we show the statistical SBR of all the 1141 neurons in both RFLFM and FLFM modes (Appendix C, Fig. S6 in Ref. [32]). The results show the median value of the SBR improvement is 15.5, which suggests a remarkable advance. For comparison, we show the reconstructed results of the flashing neurons in Fig. S7 in Ref. [32] (see also Fig. S8 in Ref. [32] and Visualization 1). We choose three planes at depths of z=30, 0, 30  μm from both the RFLFM and FLFM reconstructed volume, respectively. The enlarged ROIs (labeled with colored boxes) are shown in Figs. S7(c), S7(f), and S7(i) in Ref. [32], in which the left and medium columns show the ROIs, respectively, in RFLFM and FLFM modes. It shows that RFLFM can remove the background fluorescence effectively, thus making the system more sensitive to weak signals.

    C. High-Speed, High-Contrast Volumetric Imaging of Heart-Beating Dynamics in Larval Zebrafish in vivo

    Imaging of fast dynamics in 3D is challenging, considering the volumetric imaging speed is always limited. Here, we perform imaging of hearts beating in larval zebrafish to demonstrate that our RFLFM is competent for high-contrast fast volumetric imaging. We use runx1:GFP zebrafish (labeling the hematopoietic stem cells) at 2–4 days’ post fertilization as samples, and embed them in 1% agarose during imaging.

    To capture the flow of blood cells in the hearts, we perform imaging at 66.7 Hz (33.3 Hz in RFLFM mode; see also Fig. S9 in Ref. [32] and Visualization 2), and reconstruct the ROI of 300  μm×300  μm×90  μm in both the RFLFM and FLFM modes. As shown in Figs. 3(a) and 3(b), RFLFM improves the image contrast significantly compared to FLFM (a 3.9 times improvement). For example, the SBR of the selected cell is improved 21.3 times, as shown in Fig. 3(c). To show the capability of 3D tracking for a single blood cell, we track a blood cell and record its position at different times, as shown in Figs. 3(d) and 3(e). The cell flows into the atrium and then is pumped into the artery by the ventricle, as shown in Fig. 3(d). Successfully capturing such fast dynamics relies on the superior capability of our RFLFM in high-speed, high-contrast volumetric imaging.

    High-contrast fast volumetric imaging of heart beating in larval zebrafish in vivo. The imaging depth is centered at about tens of microns below the body surface of the zebrafish. (a) and (b) Volumetric imaging of heart beating at t=0.3 s, captured by RFLFM and FLFM, respectively. Scale bar: 50 μm. (c) Intensity distribution in blue boxes in (a) and (b), respectively. (d) 3D trajectory of a single blood cell in the heart of a larval zebrafish. (e) Tracing of the blood cell in (d) at different time points. Red circles indicate the blood cell. See also in Visualization 2.

    Figure 3.High-contrast fast volumetric imaging of heart beating in larval zebrafish in vivo. The imaging depth is centered at about tens of microns below the body surface of the zebrafish. (a) and (b) Volumetric imaging of heart beating at t=0.3  s, captured by RFLFM and FLFM, respectively. Scale bar: 50 μm. (c) Intensity distribution in blue boxes in (a) and (b), respectively. (d) 3D trajectory of a single blood cell in the heart of a larval zebrafish. (e) Tracing of the blood cell in (d) at different time points. Red circles indicate the blood cell. See also in Visualization 2.

    D. High-Contrast Volumetric Imaging of Vascular Dilations in Mouse Brains in vivo

    To show the capability of RFLFM in high-contrast volumetric imaging in more turbid tissues, we also perform structural imaging in mouse brains in vivo. We use adult C57BL/6 mice and perform craniotomy for chronic imaging. After 2–4 weeks, we inject FITC (70,000 MW, Sigma-Aldrich, MilliporeSigma) into the blood vessels (2% w/v in saline, 200 mg/kg) for staining the blood plasma [38]. Then we perform volumetric imaging of vascular dilations when the mice are under anesthesia and head-restrained under the microscope.

    The volumetric images of blood vessels achieved by RFLFM and FLFM are shown in Figs. 4(a) and 4(b), respectively. It can be seen that the image contrast is improved significantly in RFLFM. We highlight three ROIs (noted as 1–3 and 1-3) at different depths in Figs. 4(a) and 4(b), correspondingly. We show their zoom-in views and the corresponding profiles in Fig. S10 in Ref. [32]. The SBRs of the selected structures 1–3 are improved, respectively, by 40.1, 84.1, and 644.5 times. In addition, the total image contrast is improved from 0.0753 to 0.7635, which suggests that RFLFM could get rid of the background and improve image contrast by as much as 10.4 times.

    High-contrast volumetric imaging of vascular dilations in mouse cerebral cortex in vivo. The imaging depth is centered at about tens of microns below the cortical surface of mouse brains. (a) and (b) Volumetric images of blood vessels achieved by RFLFM and FLFM, respectively. Color coded depth: [−45,45] μm. Scale bar: 100 μm. (c) Enlarged ROIs in boxes 1–3 in (a), respectively. All images are normalized to themselves. Depths of ROIs: ROI 1 at 22.5 μm, ROI 2 at 0 μm, ROI 3 at 36 μm, respectively. (d) Dilations of blood vessels shown in ROIs 1–3, respectively. See also Visualization 3.

    Figure 4.High-contrast volumetric imaging of vascular dilations in mouse cerebral cortex in vivo. The imaging depth is centered at about tens of microns below the cortical surface of mouse brains. (a) and (b) Volumetric images of blood vessels achieved by RFLFM and FLFM, respectively. Color coded depth: [45,45]  μm. Scale bar: 100 μm. (c) Enlarged ROIs in boxes 1–3 in (a), respectively. All images are normalized to themselves. Depths of ROIs: ROI 1 at 22.5 μm, ROI 2 at 0 μm, ROI 3 at 36 μm, respectively. (d) Dilations of blood vessels shown in ROIs 1–3, respectively. See also Visualization 3.

    We perform volumetric imaging of vascular networks at 5 Hz. The diameters of the dilation of blood vessels, as shown in Fig. 4(c) (diameters of the blood vessels are measured using the plugin DiameterJ in ImageJ), agree with the neurovascular coupling hypothesis [3941]. From Fig. 4(d), we can see that the three selected vessels dilate with similar traces (also see Fig. S11 in Ref. [32] and Visualization 3). The capability in large-scale volumetric imaging of vascular dilations provides a potential means to study neurovascular coupling in vivo.

    E. High-Contrast Robust Volumetric Imaging of Neuronal Network Activity in Mouse Brains in vivo

    Functional imaging of neuronal network activity in mouse brains is challenging for conventional single-photon imaging techniques because it suffers from low contrast led by strong tissue scattering [3,18]. In addition, the original signal of neuronal activity may be easily overwhelmed by global and/or local background fluctuations, which causes unexcepted artifacts. Here, we employ RFLFM in high-contrast robust volumetric functional imaging in mouse brains in vivo.

    We use adult Rasgrf-dCre: Ai148 mice with GCaMP-6f expressing in cortical layers 2/3 as samples and perform craniotomy for chronic imaging. After two weeks to recuperate, we perform calcium imaging when the mice are awake and head-restrained under the microscope. We reconstruct a volume of 840  μm×840  μm×90  μm over 200 s (500 frames), as shown in Fig. 5. The normalized MIP (over time in the xyzt domain) images in both RFLFM and FLFM are shown in Figs. 5(a) and 5(b), respectively. Neurons recorded in the RFLFM mode show more clear morphology and higher contrast than those in FLFM mode (a 3.4 times improvement). We use the CNMF-E algorithm [36] for post-processing, and use its output masks to extract calcium signals. We indicate the position of every extracted neuron in Fig. 5(c) in green, and randomly select several neurons (indicated in red) for performance comparisons between the RFLFM and FLFM. Considering the weak ΔF/F in FLFM, we multiply by a factor of 10 in the FLFM results in Fig. 5(e). The orange lines show the RFLFM results and the blue lines show the enlarged FLFM results. We show the ΔF/F maps for all neurons in Figs. 5(d) and 5(f), and Fig.  S12 in Ref. [32], in both RFLFM and FLFM, respectively (also see Fig. S13 in Ref. [32] and Visualization 4). The depth of each neuron can be found by referring to Fig. S12 in Ref. [32]. Apparently, the results of FLFM are mixed with background fluctuations. By contrast, the RFLFM is robust with no apparent interference, as shown in Fig. S14 in Ref. [32]. Moreover, we perform statistics for the SBR of all neurons in both RFLFM and FLFM modes based on the extracting results (Appendix C, Fig. S15 in Ref. [32]), which suggests that the SBR is improved obviously in RFLFM. The median value of SBR improvement is 43.6, and about 30% of the values are beyond two orders of magnitude.

    High-contrast volumetric imaging of neuronal network activity in mouse cerebral cortex in vivo. The imaging depth is centered at about 150–200 μm below the cortical surface of the mouse brains. (a) and (b) MIPs over time in 4D domain (x–y–z–t), processed by RFLFM and FLFM, respectively. Scale bar: 180 μm. (c) 3D positions of post-extracted neurons. (d) and (f) Neuronal network activity based on RFLFM and FLFM, respectively. Volumetric rate in RFLFM: 2.5 Hz. (e) Activity traces of selected neurons in (c) (labeled in red). The orange lines show the RFLFM processed results, and the blue lines are the FLFM processed results enlarged 10 times. The order of each neuron is indicated in the left column. (g) and (h) Traces for the selected 20 neurons in (d) labeled yellow box and (f) labeled red box. Also see Visualization 4.

    Figure 5.High-contrast volumetric imaging of neuronal network activity in mouse cerebral cortex in vivo. The imaging depth is centered at about 150–200 μm below the cortical surface of the mouse brains. (a) and (b) MIPs over time in 4D domain (xyzt), processed by RFLFM and FLFM, respectively. Scale bar: 180 μm. (c) 3D positions of post-extracted neurons. (d) and (f) Neuronal network activity based on RFLFM and FLFM, respectively. Volumetric rate in RFLFM: 2.5 Hz. (e) Activity traces of selected neurons in (c) (labeled in red). The orange lines show the RFLFM processed results, and the blue lines are the FLFM processed results enlarged 10 times. The order of each neuron is indicated in the left column. (g) and (h) Traces for the selected 20 neurons in (d) labeled yellow box and (f) labeled red box. Also see Visualization 4.

    For further comparison, we indicate two background-induced artifacts at different moments by red and green arrows, and show the temporal traces of 20 neurons (Nos. #41–#60) in these time intervals [labeled by orange box in Fig. 5(d) and red box in Fig. 5(f), respectively] in Figs. 5(g) and 5(h), corresponding, respectively, to the RFLFM and FLFM modes. In Fig. 5(h), all neurons exhibit similar temporal trace moments indicated by the red and green backgrounds. In contrast, the neurons show independent activities in Fig. 5(g). We also analyze the relationship between background fluctuations and neuronal signals in FLFM and RFLFM, as shown by Fig. S16 in Ref. [32]. We choose two neurons (#2 and #40) as examples, and get the difference between their temporal traces in the two modes, as shown in Figs. S16(a), S16(b) and S16(d) in Ref. [32], by subtracting the normalized intensity. As neuronal activity is temporally sparse, we average the whole FOV pixels in FLFM results as the background for each frame to get an approximate background fluctuation in FLFM, as shown in Fig. S16(c) in Ref. [32]. The background fluctuation shows a high correlation with the difference of neuronal signals between FLFM and RFLFM, which strongly suggests that RFLFM can eliminate the background fluctuation-induced artifacts, but preserve the original neural activity signals.

    4. DISCUSSION AND CONCLUSIONS

    By introducing the structured illumination and computational sectioning algorithm, we propose RFLFM for robust, high-speed, high-contrast volumetric imaging in vivo. Different from former methods, there is no need for prior sample assumption and no restriction of sample size in RFLFM. Moreover, RFLFM is robust in intense background interference conditions.

    To summarize, we experimentally build an RFLFM system with 2.5–4 μm lateral resolution and 5–9 μm axial resolution in an 840  μm×840  μm×90  μm volume, which enables subcellular imaging across a large-scale volume. To demonstrate the advantage of RFLFM in high-contrast robust imaging, we perform both structural imaging and functional imaging in larval zebrafish and mouse in vivo. We verify that RFLFM can avoid the background fluctuation-induced artifacts in functional imaging of turbid brain tissues and improve the SBR by orders of magnitude. We also find that RFLFM performs better in samples of stronger scattering. Except for reducing the out of DOF background signal, RFLFM can also reject the scattered emission light by evaluating the image contrast, which shows advantages compared to the selective-volume illumination strategy [22,24]. Moreover, the optical-sectioning range can be adjusted by changing the frequency of the structured grids, which makes it flexible for different applications [4244]. We expect that our technique would find wide application in robust high-contrast volumetric imaging of biodynamics in vivo.

    Acknowledgment

    Acknowledgment. Authors Jiazhen Zhai and Ruheng Shi thank Zheng Jiang and Kuikui Fan for helping with the experimental animal preparation. Lingjie Kong thanks Tsinghua University for its support.

    APPENDIX A: SYSTEM CALIBRATION

    To calibrate the optical resolution experimentally, we use Φ=1.1  μm microfluorescent beads (Thermo Fisher Scientific) as the sample, with a size that is smaller than the diffraction-limited resolution of our optical setup. We first use a sparse sample to obtain the 3D PSF by moving the 3D stage in the z direction at the 1.5 μm step. Then, we extract the real PSF from such image stacks (shown in Figs. S17 and S18 in Ref. [32]), based on which we reconstruct the volumetric image by Richardson–Lucy deconvolution [6,25]. The reconstructed results and the statistical optical resolutions (i.e., in FWHM) are shown in Figs. S19, S20, and S21 in Ref. [32], respectively. We achieve the best optical resolution up to 2.13, 2.11, and 4.10 μm in x, y, and z direction, respectively. The lateral resolution varies between 2.5 and 4 μm, with the axial resolution between 5 and 9 μm, suggesting that the optical resolution gets worse with an increase in the defocus depth. Compared to the theoretical values of optical resolution (2.54 μm in the xy direction and 3.94 μm in the z direction (Fig. S18 and Supplementary Note 1 in Ref. [32]), our system is almost near the diffraction limit and thus is suitable for fast volumetric imaging of biodynamics.

    Furthermore, we analyze the system aberrations with Zemax simulations based on ray tracing (Fig. S22 in Ref. [32]). The results show that the maximum RMS radius induced by aberrations is about 0.634 μm (diffraction is not included), which is smaller than the diffraction limit. We also test the system’s performance in samples of different densities by numerical simulations (Fig. S23 in Ref. [32]), and it suggests that our system can work at subcellular resolution in the imaging of highly dense samples.

    APPENDIX B: DATA PROCESSING

    The procedure of image reconstruction in RFLFM is shown in Fig. S3 in Ref. [32]. Specifically, we first load the uniform pattern and structured pattern (shown in Fig. S2a in Ref. [32]) into the memory of DMD, and start the microcontroller. As the microcontroller provides rising edges, the patterns are displayed sequentially, synchronized with the exposure of the camera. After imaging, we deinterleave the raw images (amount: 2n) into two groups: one group (amount: n) is for images captured under uniform illumination, and another group (amount: n) is for images captured under structured illumination. The two groups of images are segmented to 31×n subimages, respectively. Then, we use the HiLo algorithm (ImageJ plugins: HiLo Grid, developed by Santos et al. [27], Lim et al. [28,29], and Ford et al. [31]) to achieve optical-sectioning images for all views, respectively. We prove that the HiLo algorithm can hardly influence the PSF (Fig. S24 in Ref. [32]); thus, the accuracy of the reconstruction results will not be affected. Finally, we employ the Richardson–Lucy deconvolution method [6,25] and our captured PSF to reconstruct the volumetric images in RFLFM, which costs about 13 s for one volume with the GPU (GTX 1080Ti, NVIDA) acceleration at 10 iterations.

    APPENDIX C: PERFORMANCE METRICS

    Image Contrast

    Image contrast C(ρ) is an important factor in imaging. In this paper, it is used to characterize the image quality of a whole image, which can be calculated as C(ρ)=σ[I(ρ)]I[ρ],where σ[I(ρ)] and I(ρ) represent the standard deviation and average of the selected image. The image contrast directly reflects the imaging quality, and higher contrast is desired in imaging.

    SBR

    In this paper, the SBR is used to measure the signal gains in a small area (like a neuron). It is defined as SBR=IsignalIbackgroundIbackground.

    The background intensity Ibackground of each neuron is estimated by averaging the intensity of the lowest four pixels in their neighbors (15×15 pixels for larval zebrafish, 21×21 pixels for a mouse). We compile statistics for the SBRs for all neurons and use the median value as the typical SBR.

    Pearson Coefficient

    We use the Pearson coefficient to study whether the ΔF/F detected from each neuron reflects its own activity or is overwhelmed by the background [37]. The Pearson coefficient of the two vectors X and Y is calculated as ρX,Y=cov(X,Y)σXσY,where cov means the covarions, and σX, σY are the standard derivation of X and Y, respectively. If the ΔF/F detected from individual neurons mostly reflects the change in the out-of-focus fluorescence, the Pearson correlations between pairs of neurons would be high.

    References

    [1] M. Levoy, R. Ng, A. Adams, M. Footer, M. Horowitz. Light field microscopy. ACM Trans. Graph., 25, 924-934(2006).

    [2] R. Prevedel, Y. G. Yoon, M. Hoffmann, N. Pak, G. Wetzstein, S. Kato, T. Schrodel, R. Raskar, M. Zimmer, E. S. Boyden, A. Vaziri. Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy. Nat. Methods, 11, 727-730(2014).

    [3] T. Nobauer, O. Skocek, A. J. Pernia-Andrade, L. Weilguny, F. M. Traub, M. I. Molodtsov, A. Vaziri. Video rate volumetric Ca2+ imaging across cortex using seeded iterative demixing (SID) microscopy. Nat. Methods, 14, 811-818(2017).

    [4] J. Wu, Z. Lu, D. Jiang, Y. Guo, H. Qiao, Y. Zhang, T. Zhu, Y. Cai, X. Zhang, K. Zhanghao, H. Xie, T. Yan, G. Zhang, X. Li, Z. Jiang, X. Lin, L. Fang, B. Zhou, P. Xi, J. Fan, L. Yu, Q. Dai. Iterative tomography with digital adaptive optics permits hour-long intravital observation of 3D subcellular dynamics at millisecond scale. Cell, 184, 3318-3332(2021).

    [5] H. Li, C. Guo, D. Kim-Holzapfel, W. Li, Y. Altshuller, B. Schroeder, W. Liu, Y. Meng, J. B. French, K. I. Takamaru, M. A. Frohman, S. Jia. Fast, volumetric live-cell imaging using high-resolution light-field microscopy. Biomed. Opt. Express, 10, 29-49(2019).

    [6] M. Broxton, L. Grosenick, S. Yang, N. Cohen, A. Andalman, K. Deisseroth, M. Levoy. Wave optics theory and 3-D deconvolution for the light field microscope. Opt. Express, 21, 25418-25439(2013).

    [7] N. Wagner, F. Beuttenmueller, N. Norlin, J. Gierten, J. C. Boffi, J. Wittbrodt, M. Weigert, L. Hufnagel, R. Prevedel, A. Kreshuk. Deep learning-enhanced light-field imaging with continuous validation. Nat. Methods, 18, 557-563(2021).

    [8] Y. Zhang, B. Xiong, Y. Zhang, Z. Lu, J. Wu, Q. Dai. DiLFM: an artifact-suppressed and noise-robust light-field microscopy through dictionary learning. Light Sci. Appl., 10, 152(2021).

    [9] C. Guo, W. Liu, X. Hua, H. Li, S. Jia. Fourier light-field microscopy. Opt. Express, 27, 25573-25594(2019).

    [10] A. Llavador, J. Sola-Pikabea, G. Saavedra, B. Javidi, M. Martinez-Corral. Resolution improvements in integral microscopy with Fourier plane recording. Opt. Express, 24, 20792-20798(2016).

    [11] Y. Xue, G. D. Ian, A. B. David, L. Tian. Single-shot 3D wide-field fluorescence imaging with a Computational Miniature Mesoscope. Sci. Adv., 6, eabb7508(2020).

    [12] X. Hua, W. Liu, S. Jia. High-resolution Fourier light-field microscopy for volumetric multi-color live-cell imaging. Optica, 8, 614-620(2021).

    [13] L. Cong, Z. Wang, Y. Chai, W. Hang, C. Shang, W. Yang, L. Bai, J. Du, K. Wang, Q. Wen. Rapid whole brain imaging of neural activity in freely behaving larval zebrafish (Danio rerio). eLife, 6, e28158(2017).

    [14] R. R. Sims, S. A. Rehman, M. O. Lenz, S. I. Benaissa, E. Bruggeman, A. Clark, E. W. Sanders, A. Ponjavic, L. Muresan, S. F. Lee, K. O’Holleran. Single molecule light field microscopy. Optica, 7, 1065-1072(2020).

    [15] B. J. Chang, J. D. Manton, E. Sapoznik, T. Pohlkamp, T. S. Terrones, E. S. Welf, V. S. Murali, P. Roudot, K. Hake, L. Whitehead, A. G. York, K. M. Dean, R. Fiolka. Real-time multi-angle projection imaging of biological dynamics. Nat. Methods, 18, 829-834(2021).

    [16] J. Mertz. Optical sectioning microscopy with planar or structured illumination. Nat. Methods, 8, 811-819(2011).

    [17] V. Ntziachristos. Going deeper than microscopy: the optical imaging frontier in biology. Nat. Methods, 7, 603-614(2010).

    [18] Y. Zhang, Z. Lu, J. Wu, X. Lin, D. Jiang, Y. Cai, J. Xie, Y. Wang, T. Zhu, X. Ji, Q. Dai. Computational optical sectioning with an incoherent multiscale scattering model for light-field microscopy. Nat. Commun., 12, 6391(2021).

    [19] M. A. Taylor, T. Nöbauer, A. Pernia-Andrade, F. Schlumm, A. Vaziri. Brain-wide 3D light-field imaging of neuronal activity with speckle-enhanced resolution. Optica, 5, 345-353(2018).

    [20] N. Wagner, N. Norlin, J. Gierten, G. de Medeiros, B. Balázs, J. Wittbrodt, L. Hufnagel, R. Prevedel. Instantaneous isotropic volumetric imaging of fast biological processes. Nat. Methods, 16, 497-500(2019).

    [21] D. Wang, S. Xu, P. Pant, E. Redington, S. Soltanian-Zadeh, S. Farsiu, Y. Gong. Hybrid light-sheet and light-field microscope for high resolution and large volume neuroimaging. Biomed. Opt. Express, 10, 6595-6610(2019).

    [22] Y.-G. Yoon, Z. Wang, N. Pak, D. Park, P. Dai, J. S. Kang, H.-J. Suk, P. Symvoulidis, B. Guner-Ataman, K. Wang, E. S. Boyden. Sparse decomposition light-field microscopy for high speed imaging of neuronal activity. Optica, 7, 1457-1468(2020).

    [23] Z. Fu, Q. Geng, J. Chen, L. A. Chu, A. S. Chiang, S. C. Chen. Light field microscopy based on structured light illumination. Opt. Lett., 46, 3424-3427(2021).

    [24] S. Madaan, K. Keomanee-Dizon, M. Jones, C. Zhong, A. Nadtochiy, P. Luu, S. E. Fraser, T. V. Truong. Single-objective selective-volume illumination microscopy enables high-contrast light-field imaging. Opt. Lett., 46, 2860-2863(2021).

    [25] Z. Zhang, L. Bai, L. Cong, P. Yu, T. Zhang, W. Shi, F. Li, J. Du, K. Wang. Imaging volumetric dynamics at high speed in mouse and zebrafish brain with confocal light field microscopy. Nat. Biotechnol., 39, 74-83(2021).

    [26] N. Bozinovic, C. Ventalon, T. Ford, J. Mertz. Fluorescence endomicroscopy with structured illumination. Opt. Express, 16, 8016-8025(2008).

    [27] S. Santos, K. K. Chu, D. Lim, N. Bozinovic, T. N. Ford, C. Hourtoule, A. C. Bartoo, S. K. Singh, J. Mertz. Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle. J. Biomed. Opt., 14, 030502(2009).

    [28] D. Lim, K. K. Chu, J. Mertz. Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy. Opt. Lett., 33, 1819-1821(2008).

    [29] D. Lim, T. Ford, K. Chu, J. Mertz. Optically sectioned in vivo imaging with speckle illumination HiLo microscopy. J. Biomed. Opt., 16, 016014(2011).

    [30] Q. Zhang, D. Pan, N. Ji. High-resolution in vivo optical-sectioning widefield microendoscopy. Optica, 7, 1287-1290(2020).

    [31] T. N. Ford, D. Lim, J. Mertz. Fast optically sectioned fluorescence HiLo endomicroscopy. J. Biomed. Opt., 17, 021105(2012).

    [32] J. Zhai, R. Shi, L. Kong. RFLFM supplementary information.

    [33] D. Dan, M. Lei, B. Yao, W. Wang, M. Winterhalder, A. Zumbusch, Y. Qi, L. Xia, S. Yan, Y. Yang, P. Gao, T. Ye, W. Zhao. DMD-based LED-illumination super-resolution and optical sectioning microscopy. Sci. Rep., 3, 1116(2013).

    [34] R. Shi, L. Kong. Evaluating structured-illumination patterns in optimizing optical-sectioning of HiLo microscopy. J. Phys. D, 54, 414001(2021).

    [35] H. W. Detrich, H. M. Jung, S. Isogai, M. Westerfield, L. I. Zon, M. Kamei, D. Castranova, A. V. Gore, B. M. Weinstein. Chapter 4 - Imaging blood vessels and lymphatic vessels in the zebrafish. Methods in Cell Biology, 69-103(2016).

    [36] P. Zhou, S. L. Resendez, J. Rodriguez-Romaguera, J. C. Jimenez, S. Q. Neufeld, A. Giovannucci, J. Friedrich, E. A. Pnevmatikakis, G. D. Stuber, R. Hen, M. A. Kheirbek, B. L. Sabatini, R. E. Kass, L. Paninski. Efficient and accurate extraction of in vivo calcium signals from microendoscopic video data. elife, 7, e28728(2018).

    [37] E. Yoshida, S. I. Terada, Y. H. Tanaka, K. Kobayashi, M. Ohkura, J. Nakai, M. Matsuzaki. In vivo wide-field calcium imaging of mouse thalamocortical synapses with an 8 K ultra-high-definition camera. Sci. Rep., 8, 8324(2018).

    [38] F. Appaix, S. Girod, S. Boisseau, J. Römer, J.-C. Vial, M. Albrieux, M. Maurin, A. Depaulis, I. Guillemain, B. van der Sanden. Specific in vivo staining of astrocytes in the whole brain after intravenous injection of sulforhodamine dyes. PLoS ONE, 7, e35169(2012).

    [39] H. Girouard, C. Iadecola. Neurovascular coupling in the normal brain and in hypertension, stroke, and Alzheimer disease. J. Appl. Physiol., 100, 328-335(2006).

    [40] J. H. Park, L. Kong, Y. Zhou, M. Cui. Large-field-of-view imaging by multi-pupil adaptive optics. Nat. Methods, 14, 581-583(2017).

    [41] J. L. Fan, J. A. Rivera, W. Sun, J. Peterson, H. Haeberle, S. Rubin, N. Ji. High-speed volumetric two-photon fluorescence imaging of neurovascular dynamics. Nat. Commun., 11, 6020(2020).

    [42] D. Dana, M. D. Walter, S. John. Emerging digital micromirror device (DMD) applications. Proc. SPIE, 4985, 14-25(2003).

    [43] D. H. Kim, J. Kim, J. C. Marques, A. Grama, D. G. C. Hildebrand, W. Gu, J. M. Li, D. N. Robson. Pan-neuronal calcium imaging with cellular resolution in freely swimming zebrafish. Nat. Methods, 14, 1107-1114(2017).

    [44] R. Shi, C. Jin, H. Xie, Y. Zhang, X. Li, Q. Dai, L. Kong. Multi-plane, wide-field fluorescent microscopy for biodynamic imaging in vivo. Biomed. Opt. Express, 10, 6625-6635(2019).

    Jiazhen Zhai, Ruheng Shi, Lingjie Kong. Improving signal-to-background ratio by orders of magnitude in high-speed volumetric imaging in vivo by robust Fourier light field microscopy[J]. Photonics Research, 2022, 10(5): 1255
    Download Citation