Volumetric imaging provides a comprehensive understanding of the three-dimensional (3D) structure of the sample^[1–3], both internally and externally. Light field microscopy^[4], a scanless volumetric imaging technique, captures the two-dimensional (2D) spatial information and the 2D angular information of the light passing through the sample volume. Post-processing of this spatial-angular information enables the reconstruction of the volumetric images of the sample. In dynamic sample volumetric imaging, temporal resolution is also important. However, obtaining the full five-dimensional (5D) light field—encompassing 2D spatial, 2D angular, and one-dimensional (1D) temporal information—using a 2D pixelated detector presents challenges, necessitating strategic trade-offs. Figure 1 illustrates three primary trade-off strategies used in light field microscopy: microlens array (MLA)-based^[4], image-coding-based^[5–7], and aperture-coding-based approaches^[8–10].

The MLA-based light field microscopy offers high temporal resolution but is limited by a trade-off between spatial and angular resolution^[11–13], as shown in Fig. 1(a). Spatial resolution is crucial in microscopy, and the low spatial resolution of the MLA-based light field microscope restricts its applications. Although moving the MLA from the image plane to the pupil plane of the objective lens, as in Fourier light field microscopy^[14–18], can alleviate the low spatial resolution problem, it does not fundamentally resolve the spatial-angular resolution trade-off. Recently, scanning light field microscopy^[19,20] was proposed to address the trade-off between angular and spatial resolution existing in traditional light field microscopy. However, this method requires a high-precision 2D mechanical scanning system.

The image-coding-based light field microscopy^[5–7], as shown in Fig. 1(b), offers high angular resolution but suffers from a trade-off between spatial resolution and temporal resolution. It uses a spatial light modulator (SLM) to generate structured patterns for coding the sample image at the image plane, capturing the Fourier spectral images representing the angular information of the light at the Fourier plane through multiple exposures. These captured Fourier spectral images retain the full sampling resolution of the camera, facilitating the acquisition of high angular resolution light field images^[21]. Furthermore, each pixel of the camera acts as a single-pixel detector, enabling sample image reconstruction via the single-pixel imaging method^[22,23], based on the principle of dual photography^[24]. The spatial resolution of the reconstructed image of the sample matches the sampling resolution of the SLM, allowing for high spatial-resolution imaging. However, this method requires a large number of measurements for high spatial resolution image reconstruction, leading to a trade-off between temporal resolution and spatial resolution and making it unsuitable for imaging dynamic samples with high spatial resolution.

Aperture-coding-based light field microscopy^[8–10], as shown in Fig. 1(c), offers high spatial resolution but suffers from a trade-off between temporal resolution and angular resolution. It uses an SLM to code the aperture at the Fourier plane and capture sample images at the image plane with multiple exposures, maintaining the full sampling resolution of the camera for high spatial resolution light field imaging. The angular resolution of the light field matches the sampling resolution of the SLM. Although aperture-coding-based light field microscopes can obtain light field images with a high angular resolution by coding the aperture with an SLM of high spatial sampling resolution, they require a large number of measurements, resulting in reduced temporal resolution. Consequently, this method faces a trade-off between temporal resolution and angular resolution, making it unsuitable for imaging dynamic samples, either.

To address these challenges in light field imaging for dynamic microscopic samples, we propose a high-speed differential aperture-coding light field microscopy. This method uses a high-speed SLM and a high-speed camera to accelerate the aperture coding rate and the image acquisition rate. Additionally, it uses the undersampling strategy to enhance the temporal resolution without compromising the depth of field (DOF) in light field imaging, all without the need for iterative optimization. Experimental results have shown that our approach can capture the dynamics of swimming marine planktonic samples in volumes of $\sim 1518\mathrm{\mu m}\times 1172\mathrm{\mu m}\times 600\mathrm{\mu m}$ at 41 Hz, under the condition of an SLM refresh rate of 1340 Hz and a camera frame rate of 1340 frame/s, using an objective lens with a numerical aperture of 0.3 and a magnification of 10.

Image contrast, alongside spatial resolution, is a critical parameter in microscopic imaging. The unavoidable direct current (DC) background reduces contrast in an aperture-coding-based light field microscopy image. Conventional aperture-coding-based light field microscopes typically use the S-matrix-coded patterns^[25]. The S-matrix is a cyclic matrix where each row is derived by shifting the previous row one place to the left in a cyclical manner^[26]. However, the image reconstruction algorithm based on S-matrix cannot eliminate the DC background. We have used a differential coding mechanism^[27] to achieve the DC background elimination. The proposed method holds great promise for widespread application in the volumetric imaging of dynamic microscopic samples.

The proposed method exploits dual photography^[24] to reconstruct light field images. Figure 2 illustrates the schematic diagram of the proposed system. It should be noted that the optical path shown in Fig. 2 is slightly different from that given in dual photography^[24]. In dual photography, it is real-space duality. The modulation plane of the SLM and the photosensitive plane of the camera are conjugate to each other. However, in Fig. 2, the modulation plane of the SLM is located in the Fourier plane, and it is Fourier duality. Figure 2(a) is the primal configuration of the imaging system, where an SLM is placed at the back focal plane (aperture plane or Fourier plane) of the Fourier lens to code the Fourier spectral image and a camera captures the image of the sample at the image plane. The captured images can be used to reconstruct the Fourier spectral image at the modulation plane of the SLM via single-pixel imaging. Single-pixel imaging is subject to Helmholtz reciprocity^[24,28]. The role of the SLM shown in Fig. 2(a) is equivalent to the role of the virtual camera, and the role of the camera is equivalent to the role of the virtual SLM in the dual configuration of the imaging system shown in Fig. 2(b). The Fourier spectral image reconstructed by single-pixel imaging is equivalent to the image recorded by the virtual camera under the modulation of the virtual SLM shown in Fig. 2(b).

The Fourier spectral image contains angular information of the light field. Therefore, we can sample the spatial information with the real camera and the angular information with the virtual camera, respectively. The spatial resolution of the image captured by the real camera maintains the full sampling resolution of the real camera. The angular resolution (namely the Fourier spectral image’s spatial resolution) matches the SLM’s sampling resolution. The sampling resolution of the SLM can be adjusted digitally by changing the size of the modulation unit. A smaller size provides higher sampling resolution but decreases the temporal resolution due to the increased number of images captured by the camera.

We use a high-speed camera and a high-speed SLM to enhance the temporal resolution and realize the dynamic light field microscopic imaging to accelerate the image acquisition rate and modulation rate. Furthermore, to subtract the DC background from the image captured by the camera and reconstruct high-contrast light field images, we propose a differential aperture-coding mechanism and use each pixel of the camera as a single-pixel detector to reconstruct a Fourier spectral image by single-pixel imaging.

Many single-pixel imaging methods have been developed^[28,29]. We use a Fourier single-pixel imaging method^[22] with a differential measurement mechanism to eliminate the DC background and reconstruct high-contrast light field images because Fourier single-pixel imaging is more efficient than other single-pixel imaging methods^[23]. We use an SLM to load a set of phase-shifting Fourier basis patterns to perform differential coding for the Fourier spectral image at the Fourier plane by Fourier single-pixel imaging. The phase-shifting Fourier basis pattern can be expressed as $$P(u,v;f_u,f_v,\varphi )=a+b\cos [2\pi (f_{u}u+f_{v}v)+\varphi ],$$(1)where $(u,v)$ represents the spatial coordinates of the modulation plane of the SLM, $(f_u,f_v)$ represents the spatial frequency, $a$ represents the background, $b$ represents the modulation, $\varphi =2\pi k/3$ represents the phase shift amount, and $k=0,1,2$ represent the phase shift numbers.

After coding the Fourier spectral image, we use the camera to capture the image of the sample. The intensity recorded by each pixel of the camera can be expressed as $$D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;\varphi )=\epsilon \iint I_{(x_{\mathrm{c}},y_{\mathrm{c}})}(u,v)P(u,v;f_u,f_v;\varphi )\mathrm{d}u\mathrm{d}v+n_{(x_{\mathrm{c}},y_{\mathrm{c}})}^d,$$(2)where $(x_{\mathrm{c}},y_{\mathrm{c}})$ represents the camera pixel coordinates, $\epsilon$ is a factor that depends on the size and position of the detector, $I_{(x_{\mathrm{c}},y_{\mathrm{c}})}(u,v)$ represents the Fourier spectral image at the Fourier plane, and $n_{(x_{\mathrm{c}},y_{\mathrm{c}})}^d$ represents the background light intensity recorded by the camera pixel $(x_{\mathrm{c}},y_{\mathrm{c}})$.

When we use the SLM to load a set of three-step phase-shifting Fourier basis patterns with a frequency of $(f_u,f_v)$ and phase difference of $2\pi /3$ to code the spectrum of the sample image, we can record three intensity values $D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;0)$, $D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;2\pi /3)$, and $D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;4\pi /3)$ from each pixel of the camera. Based on the recorded three intensity values, we can calculate the Fourier coefficient $F_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v)$ of frequency $(f_u,f_v)$ using the three-step phase-shifting algorithm^[30], $$\begin{gathered} F_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v)=[2D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;0)-D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;2\pi /3)-D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;4\pi /3)] \\ +\sqrt{3}\mathit{j}·[D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;2\pi /3)-D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;4\pi /3)] \\ =2b\epsilon \iint I_{(x_{\mathrm{c}},y_{\mathrm{c}})}(u,v)e^{-j2\pi (f_{u}u+f_{v}v)}\mathrm{d}u\mathrm{d}v. \end{gathered}$$(3)

Equation (3) contains a differential mechanism that allows the DC background to be subtracted from the image captured by the camera.

By changing the spatial frequency of the Fourier basis pattern and using the above method, we can obtain the Fourier coefficients corresponding to other frequencies and construct a Fourier spectrum ${\widehat{I}}_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v)$. Because the Fourier spectrum of natural images is conjugate symmetry, we only need to obtain half the number of the Fourier coefficients. The remaining half of the number of the Fourier coefficients can be obtained using conjugate symmetry.

By performing the inverse Fourier transform on ${\widehat{I}}_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v)$, we can reconstruct an image, $$I_{(x_{\mathrm{c}},y_{\mathrm{c}})}(u,v)=F^{-1}\{{\widehat{I}}_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v)\}=F^{-1}\left\{\begin{array}{l}[2D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;0)-D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;2\pi /3)-D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;4\pi /3)]\\ +\sqrt{3}\mathit{j}·[D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;2\pi /3)-D_{(x_{\mathrm{c}},y_{\mathrm{c}})}(f_u,f_v;4\pi /3)]\end{array}\right\},$$(4)where $I_{(x_{\mathrm{c}},y_{\mathrm{c}})}(u,v)$ represents the Fourier spectral image of the sample image reconstructed by the camera pixel $(x_{\mathrm{c}},y_{\mathrm{c}})$. $F^{-1}\{\}$ represents the inverse Fourier transform operator. If the camera has $M_{\mathrm{c}}\times N_{\mathrm{c}}$ pixels, then using the above method, we can reconstruct $M_{\mathrm{c}}\times N_{\mathrm{c}}$ Fourier spectral images.

Assume the size of the Fourier basis patterns is $M_u\times N_v$ pixels, which is equal to the number of the modulation unit of the SLM. The size of the reconstructed Fourier spectral image $I_{(x_{\mathrm{c}},y_{\mathrm{c}})}(u,v)$ is also $M_u\times N_v$ pixels according to the principle of single-pixel imaging. Every pixel on the Fourier spectral image $I_{(x_{\mathrm{c}},y_{\mathrm{c}})}(u,v)$ represents a different angle of light.

Using the intensity values of the same pixel of all reconstructed Fourier spectral images, we can reconstruct a perspective image of the sample. This perspective image is equivalent to the image captured by the camera when we only open the point corresponding to this pixel on the Fourier plane. The size of the reconstructed Fourier spectral image is $M_u\times N_v$ pixels; thus, we can reconstruct $M_u\times N_v$ perspective images. If the camera has $M_{\mathrm{c}}\times N_{\mathrm{c}}$ pixels, each perspective image has $M_{\mathrm{c}}\times N_{\mathrm{c}}$ pixels.

Using the shift-and-sum algorithm^[31] to process the perspective images, we can reconstruct the digitally refocused images of the sample at different depths. The size of the digitally refocused images is $M_{\mathrm{c}}\times N_{\mathrm{c}}$ pixels, which is equal to the full sampling resolution of the camera.

As illustrated in Fig. 3, the experimental setup was constructed using a Nikon Eclipse 80i microscope. This microscope incorporated a collector lens, a field lens, an aperture diaphragm, a condenser lens, an objective lens, and a tube lens. To realize light field imaging, the aperture diaphragm is fully opened. The objective lens had a magnification of 10 and a numerical aperture of 0.3. The microscope produced an enlarged image of the sample in the back focal plane of the tube lens. A reflecting mirror was strategically positioned at the light exit of the microscope to direct the image of the sample. Subsequently, a Fourier lens with a focal length of 150 mm was employed to convert the image into the Fourier spectral image. To encode the Fourier spectral image, a liquid crystal on a silicon spatial light modulator (LCOS-SLM) (Four Dimension Displays, model: SXGA-R12-STR, 1280 pixel × 1024 pixel, pixel size: 13.62 µm, maximum refresh rate: 3.2 kHz) was placed in the Fourier plane. Furthermore, a polarizing beam-splitter and an imaging lens, with a focal length of 150 mm, were placed in front of the LCOS-SLM to form an image of the sample on the image plane.

We conducted four experiments to verify the proposed method. The first experiment was to show how our proposed method works and compare it with aperture-coding-based light field microscopy using S-matrix.

Figure 4 shows the process of reconstructing the perspective images using our proposed method. The imaged sample was the marine copepod Tigriopus japonicus at the copepodite stage, which originated from the Zooplankton Laboratory of Jinan University. The average size of copepods was $\sim 628.25\mathrm{\mu m}\left(\mathit{L}\right)\times 214\text{.}5\mathrm{\mu m}\left(\mathit{W}\right)\times 200\mathrm{\mu m}\left(\mathit{D}\right)$. The reconstruction utilized 53 Fourier basis patterns, each with a size of 7 pixel × 5 pixel, to code the Fourier spectral image in the Fourier plane at a 100% sampling rate for single-pixel imaging. Figure 4(a) shows a set of images captured by the camera (GS3-U3-51S5C-C, resolution: 2448 pixel × 2048 pixel, pixel size: 3.45 µm) after encoding the Fourier spectral image. From these captured images, we extracted the value from the same pixel point to form a 1D intensity sequence. Figures 4(b1)–4(b3) show the 1D intensity sequences recorded by the camera pixels (1858,465), (1222,891), and (592,1365), respectively.

Based on these 1D intensity sequences, we reconstructed the Fourier spectral images using the Fourier single-pixel imaging method. Figures 4(c1)–4(c3) show three such reconstructed Fourier spectral images. In our experiment, we grouped adjacent 120 pixel × 120 pixel of the LCOS-SLM into one modulation unit, employing a total of $7\times 5$ modulation units to load the Fourier basis patterns for coding the Fourier spectral image. Consequently, the reconstructed Fourier spectral image had 7 pixel × 5 pixel, following the principle of single-pixel imaging.

To facilitate the installation of the LCOS-SLM, we turned the modulation plane of the LCOS-SLM by 90 degrees, resulting in the Fourier spectral images shown in Figs. 4(c1)–4(c3) having 5 pixels in the horizontal direction and 7 pixels in the vertical direction. The green, red, and orange boxes in the figures represent the intensity values at the pixel coordinates (6,2), (3,4), and (5,4), respectively. By extracting the value of the same pixel point from the reconstructed Fourier spectral images and rearranging them according to the camera pixel coordinates, we reconstructed the perspective images (with the size of 2448 pixel × 2048 pixel). Figure 4(d1) presents the perspective image constructed by extracting the intensity value at the pixel coordinate (6,2) from the spectral image. Figure 4(d2) shows the perspective image constructed by extracting the intensity value at the pixel coordinate (3,4). Figure 4(d3) gives the perspective image constructed by extracting the intensity value at the pixel coordinate (5,4). Figures 4(e1)–4(e3) are the magnified views of Figs. 4(d1)–4(d3). As seen, the images of the sample at different depths exhibit lateral shifts in the perspective images.

Based on the perspective images, we reconstructed the digitally refocused images (with the size of 2448 pixel × 2048 pixel) using the shift-and-sum algorithm^[31]. For comparative analysis, we also used aperture-coding-based light field microscopy with a S-matrix to image the sample. The sizes of the modulation unit and the S-matrix encoded patterns were consistent with those used in our proposed method.

Figure 5 shows the results obtained by the two methods mentioned above. Figures 5(a) and 5(d) show the maximum intensity projections (MIPs) of a 28-image stack spanning a depth range obtained by the two methods. Figures 5(b1)–5(b3) and 5(e1)–5(e3) show the optical-sectioning results obtained by the two methods. Figures 5(c1)–5(c3) and 5(f1)–5(f3) show the zoomed-in view of the results. As can be seen, our method provides clearer details compared to the aperture-coding-based light field microscopy using the S-matrix. This enhanced clarity primarily stems from our proposed method’s differential coding mechanism, which effectively eliminates the DC background, thereby enhancing the contrast of the resultant images. It should be noted that the images obtained by our method were not subjected to any subsequent contrast-enhancing image processing algorithms.

In terms of computational efficiency, our method, when executed on a computer equipped with an Intel(R) Core (TM) i7–9700 K CPU at 3.60 GHz and 40.0 GB memory, requires $7.989\times {10}^{-5}\mathrm{s}$ to reconstruct a Fourier spectral image with the size of 7 pixel × 5 pixel. In contrast, the aperture-coded-based light field microscopy using the S-matrix requires $14.776\times {10}^{-5}\mathrm{s}$ for the reconstruction of a spectrum image of the same size. Consequently, the reconstruction speed of our proposed method is faster than that of aperture-coded-based light field microscopy using the S-matrix.

In the second experiment, we aimed to demonstrate that our method is capable of reconstructing light field images without compromising spatial resolution and surpassing the DOF that is achievable by traditional wide-field microscopy. Additionally, we quantitatively compared the contrast of the results reconstructed by our proposed method with those obtained using aperture-coding-based light field microscopy employing the S-matrix. The sample used was a standard USAF resolution target (2015a, Ready Optics). We moved the target along the optical axis direction to different positions, capturing images at each position using wide-field microscopy, aperture-coding-based light field microscopy with the S-matrix, and our proposed method.

Figure 6 shows a comparison of results obtained by the three methods when the standard resolution target was positioned at different focal depths, with experimental parameters identical to those of the previous experiment. Figures 6(d1)–6(d3) are the zoomed-in views of Figs. 6(a4)–6(c4). As seen, our proposed method and wide-field microscopy were capable of clearly resolving the 9.3th level pattern on the standard resolution target. This indicates that the lateral resolution of our proposed method is 0.775 µm. These results validate that our method can reconstruct light field images without sacrificing spatial resolution.

The axial resolution of our proposed method is determined by the smallest refocusing step, which can be calculated using the following formula^[32]: $${\delta }_z=\frac{\mathrm{\Delta }\cot [\arcsin (\mathrm{NA})]}{M},$$(5)where $\mathrm{NA}=0.3$ is the numerical aperture of the objective lens used; $M=11$ is the magnification of the proposed light field microscope, which is calibrated with the standard resolution target. $\mathrm{\Delta }=0.775\times 11=8.525\mathrm{\mu m}$ is the lateral resolution on the image plane. Based on the aforementioned parameters, using Eq. (5), we can calculate that the axial resolution of our proposed method is 2.46 µm.

Moreover, the reconstructed image contrast achieved by our method exceeds that of the aperture-coding-based light field microscopy using the S-matrix, as illustrated in Fig. 6(e). The contrast $C(\rho )$ in Fig. 6(e) was calculated by $C(\rho )=\frac{\sigma [I(\rho )]}{⟨I(\rho )⟩}$^[33], where $\sigma [I(\rho )]$ and $⟨I(\rho )⟩$ represent the standard deviation and average of the selected image, respectively. As seen in Fig. 6(e), the contrast of the results obtained by our proposed method is about 2 times that of the S-matrix method. This enhancement in contrast is primarily attributed to the effective removal of the DC background from the perspective images in our approach, a step not performed in aperture-coding light field microscopy based on the S-matrix.

Figure 6(f) provides a comparative assessment of the DOF, based on the results from the standard resolution target obtained using the three methods. The highest-resolution data were extracted and plotted to demonstrate the DOF. The results obtained by our proposed method and aperture-coding-based light field microscopy using the S-matrix exhibit a greater DOF than traditional wide-field microscopy. For example, at the resolution of 5 µm, the DOF for wide-field microscopy is 200 µm (from $-100$ to $-100\mathrm{\mu m}$). In contrast, our proposed method and aperture-coding light field-based microscopy using the S-matrix provide a deeper DOF of 450 µm (from $-200$ to 250 µm).

The third experiment demonstrated that the proposed method allows us to adjust the DOF by changing the number of measurements. Here changing the number of measurements was realized by changing the sampling rate^[23] of the reconstructed Fourier spectral images (angular information). The size of the reconstructed Fourier spectral image was 5 pixel × 5 pixel. Figure 7 shows the results of the USAF resolution target at different depths reconstructed by changing the sampling rates.

As shown in Fig. 7, reducing the sampling rate reduces the DOF. This phenomenon can be primarily attributed to the influence of the sampling rate on the resolution of the Fourier spectral image reconstructed, which indicates the angular resolution within the reconstructed light field. Consequently, a decrease in the sampling rate leads to a corresponding reduction in the DOF.

However, upon closer examination of Figs. 7(e1)–7(e9) and 7(f), we observe that the contrast and DOF of images derived from sampling rates of 100% and 60% are comparable. This is attributed to the application of Fourier single-pixel imaging, which captures angular information by extracting its frequency spectrum and subsequently reconstructing the image through an inverse Fourier transform. The angular information is predominantly concentrated in the lower frequency bands, which accounts for the comparable DOF and contrast observed across both sampling rates. To reconstruct the Fourier spectral images at a size of 5 pixel × 5 pixel, it necessitates 38 measurements at a 100% sampling rate and 25 at a 60% sampling rate. No iterative optimization is required in the reconstruction process. These findings show that the proposed method can enhance the temporal resolution of light field imaging by slightly reducing the sampling rate without sacrificing the contrast and DOF.

In the fourth experiment, we aimed to demonstrate the capability of our proposed method in observing dynamic samples. We used a high-speed camera (SH3-103, full resolution: $1280\times 1024$, full frame rate: 3000 frame/s, pixel size: 14.6 µm) to enhance the image acquisition speed. The images obtained in the experiment have been cropped, and the size is 1144 pixel × 883 pixel. The FOV of the captured image was $\sim 1518\mathrm{\mu m}\times 1172\mathrm{\mu m}$. Figure 8 shows the light field digitally refocused results of the dynamic sample. The measured samples were swimming T. japonicus copepodids. Each set of 120 pixel × 120 pixel of the SLM was bundled as a modulation unit. We used $7\times 5$ modulation units to load the Fourier basis pattern to code the Fourier spectral image. According to the principle of single-pixel imaging, the size of the reconstructed Fourier spectral image was 7 pixel × 5 pixel. For comparative analysis, we also employed conventional wide-field microscopy to capture images of the sample. Both the refresh rate of the LCOS-SLM and the frame rate of the camera were synchronized at 1340 Hz.

Figures 8(a1)–8(e1) show images of the two swimming copepods at different time points obtained by traditional wide-field microscopy. In contrast, Figs. 8(a2)–8(e2) show the results of the same sample, digitally refocused at depths with a 100% sampling rate, achieved through our proposed method. Figures 8(a3)–8(e3) show the results of the dynamic sample at different time points, refocused at the same depth, still using our method, but with the sampling rate reduced to 60%. Visualization 1 comprehensively compares the dynamic imaging results obtained by different methods.

Visualization 2 and Visualization 3 also present a comparative analysis of the dynamic light field imaging results of swimming copepod samples reconstructed using our proposed method and conventional wide-field microscopy. But different from Visualization 1, the results shown in Visualization 2 and Visualization 3 were reconstructed by employing $8\times 8$ modulation units, with each unit consisting of 75 pixel × 75 pixel of the SLM. While Visualization 2 operates at a 100% sampling rate, Visualization 3 is captured at a reduced sampling rate of 60%. Additionally, the sample preparation method diverges from conventional procedures. In this instance, we simply placed a droplet of the copepod-containing solution onto a microscope slide without the application of a coverslip.

Our experimental results indicate that the reconstructed images from our method maintain comparable DOFs, regardless of the sample rate of 100% and 60%. When imaging is performed using an objective lens with a magnification of 10 and a numerical aperture of 0.3, the resulting volume field is $\sim 1518\mathrm{\mu m}\times 1172\mathrm{\mu m}\times 600\mathrm{\mu m}$, and the DOF surpasses traditional wide-field microscopy. It is noteworthy that, when the sampling rate is 100%, 53 measurements are required for the volume reconstruction. However, by reducing the sampling rate to 60%, the number of measurements can be reduced to 32. With both the camera and the SLM operating at a refresh rate of 1340 Hz, the volume reconstruction rate is 25 Hz at a 100% sampling rate. Decreasing the sampling rate to 60% increases this reconstruction rate to 41 Hz. This finding highlights the potential for optimizing the imaging speed by adjusting the sampling rate, without any compromise in image quality.

In our experiments, we intentionally did not operate the camera and the SLM at their maximum frame rates. This strategic choice stemmed from the practical challenges of preparing microscopic dynamic samples for high-speed motion observation. The preparation of dynamic samples requires specialized immobilization techniques. These techniques must prevent sample damage and any unnatural alteration of their motion patterns. Additionally, when observing the high-speed motion of biological samples, it is crucial to ensure their viability throughout the observation period. This often involves specialized culture or preservation conditions to sustain their biological activity. Addressing these challenges typically requires interdisciplinary knowledge and skills, spanning fields such as biology, physics, optical engineering, and computer science. With these considerations in mind, we set the frame rates of the camera and SLM at 1340 Hz, rather than at their maximum settings. We are confident that with advancements in sample preparation techniques, the imaging speed of our method could be further enhanced, potentially transforming the landscape of high-speed volumetric imaging in the study of dynamic biological processes.

We note that aperture-coding light field microscopy can adjust the DOF and temporal resolution by changing the size of the modulation unit. Smaller modulation units can extend the DOF but at the expense of temporal resolution. Our experimental results show that slightly reducing the sampling rate in single-pixel imaging maintains the DOF of the light field imaging while enhancing the temporal resolution. The reconstruction process does not require any iterative optimization, providing a new method to improve the temporal resolution of light field imaging.

Theoretically, aperture-coding-based light field microscopy using the S-matrix cannot employ a sub-Nyquist sampling strategy for image reconstruction as it is analogous to solving a system of linear equations where the number of equations must at least equal the number of unknowns. When using the sub-Nyquist sampling strategy, the number of equations is less than the number of unknowns, which turns image reconstruction into solving the underdetermined problem mathematically. Although the underdetermined problem can be resolved using compressive sensing theory^[10,34,35], it often requires an iterative operation, thereby increasing the computational cost and image reconstruction time.

The imaging DOF of our method is related to angular resolution, while the angular resolution in our experiments is related to the size of the modulation unit of the LCOS-SLM. The smaller the modulation unit size, the higher the angular resolution and the deeper the DOF of light field imaging. However, a smaller modulation unit size requires more coding times and leads to a lower temporal resolution. In practical applications, the thickness of the samples and the demand for imaging depths are different. We can set the size of the modulation unit according to the actual imaging depth requirements and adjust the angular resolution.

While the use of high-speed SLMs and cameras does increase costs, high-speed SLMs and cameras are already widely used in various industrial applications. Therefore, we anticipate that the proposed method will become increasingly practical and accessible in the future.

Our proposed method is expected to improve the accuracy and speed of plankton identification and classification. Its high temporal resolution allows for precise tracking of the rapid movements of micro- and meso-zooplankton, facilitating non-invasive, real-time studies of their behavior in natural settings. This breakthrough is crucial for monitoring plankton responses to environmental changes, enhancing our understanding of marine biodiversity and its sensitivity to climate and human activities. Importantly, our proposed method offers a means for the accurate recording and observation of dynamic biological processes, thus providing a valuable tool for researchers in the field.

We proposed light field microscopy based on high-speed differential aperture-coding. Experimental results demonstrate that our approach can achieve a greater DOF than that of traditional wide-field microscopy. In addition, the proposed method can reconstruct light field images without sacrificing spatial resolution. Compared with existing aperture-coding light field microscopy, the contrast of the results reconstructed by the proposed method is higher due to the use of a differential coding mechanism. Furthermore, our method can use the undersampling strategy to enhance the temporal resolution without sacrificing the imaging quality, and no iterative optimization is needed in the reconstruction process. The proposed method paves the path for the reconstructtion of high spatial resolution and high-contrast volumetric images for dynamic samples. It is expected to be widely used in the biomedical field.