Two- or multi-viewpoint images are generally required in daily life to detect 3D information without scanning or structure illumination. Holography1 is a 3D image-recording technique with an image sensor, unlike general imaging techniques. On the one hand, a laser light source is generally adopted to realize holography because light interference is required to record 3D information in holography. On the other hand, an image of interference fringes, such as a Newton ring, can be generated with daily-use light based on the self-interference phenomenon. Incoherent holography2^–5 exploits the self-interference phenomenon and has realized the holographic recording of an object with a daily use light source such as a lamp6^,7 and a light-emitting diode.8^,9 Incoherent holography has the following attractive features: (1) The holographic 3D imaging of self-luminous light can be achieved and has been demonstrated,10 (2) lensless 3D imaging is possible without a priori acquisition of a point-spread function (PSF),11 (3) speckle-noiseless holographic 3D imaging can be conducted, (4) PSF in the in-plane direction can be improved in comparison with other incoherent imaging technologies,12 and (5) a 3D scene with a wide range of depths that are much larger than the coherence length of light is recorded as a hologram. Incoherent holography has been applied to fluorescence microscopy,10^,12^–16 tomography,17 and holographic imagery, as well as cameras.8^,9^,18^–22 In particular, Kim has succeeded in developing a full-color natural light holographic camera by adopting a Michelson interferometer.8 However, multiple exposures are required in Kim’s camera; therefore, it has been difficult to instantaneously obtain a natural-light full-color hologram of a 3D scene. To realize single-shot incoherent holography, single-shot phase-shifting (SSPS) interferometry23^–25 has been adopted for incoherent holography.19^–21 In SSPS incoherent digital holography, Fresnel incoherent correlation holography (FINCH)-based,19 phase grating-based,20 and geometric phase lens (GPL)-based21 optical systems have been proposed. In these optical systems, a diffractive optical element (DOE) is set to generate two object waves that are required for self-interference. Undesired-order diffraction waves appear during the recording of a self-interference hologram when white light such as natural light illuminates a DOE because the diffraction efficiency of a DOE is not perfect for the whole wavelength band of natural light. Instead of a DOE, the use of birefringent materials has been conceptually proposed to prevent the appearance of undesired light waves during recording and to generate an incoherent hologram of spatially and temporally incoherent light.26

We propose a method of full-color, scan-free, and single-shot natural-light holography for full-color 4D (3D + time) imaging and develop a portable natural-light motion-picture holographic camera that can be set on a movable table without any antivibration structure. We exploit SSPS incoherent digital holography,19^–21 which is based on SSPS interferometry23^–25 and incoherent holography, and design and implement the optical system adopting birefringent materials and a camera lens. Moreover, state-of-the-art denoising algorithms are applied to the proposed holography to enhance the quality of the reconstructed image. We perform full-color scan-free natural-light 3D imaging using the designed portable holographic camera set on a wooden table and the full-color 4D imaging of a moving object. This holographic camera is capable of full-color 4D imaging of objects ranging in size from the centimeter order to the 10-m order (dolls, human faces, and buildings), opening up a new stage in holographic imaging and overcoming the limitations of conventional holographic imaging despite the portability of this camera.

Figure 1 shows the basic concept of natural-light full-color motion-picture holography. The optical configuration is based on the combination of a polarimetric SSPS self-interference interferometer19 and a color polarization image sensor. Instead of a polarimetric DOE displayed on a spatial light modulator,19 a birefringent dual-focus lens such as a crystal lens used in colonoscopic holography27 is adopted. Two object waves whose wavefront curvature radii are different and whose polarization directions are orthogonal to each other are generated by the lens from the incident object wave. The optical path length difference generated by the lens is adjusted using a birefringent phase plate such as a calcite and a liquid crystal. This adjustment enables the generation of an interference fringe image of temporally incoherent light. A quarter-wave plate converts linearly polarized object waves into circularly polarized ones. The two circularly polarized object waves whose polarization directions are orthogonal to each other illuminate a color polarization image sensor in which a color filter array and a micropolarizer array are attached to a photodetector array. Each cell of the color-filter array selectively transmits light at either the red-, green- or blue-wavelength (RGB) band. The transmission axis of each cell of the micropolarizer array $\theta$ is either 0, 45, 90, or 135 deg against the horizontal axis. After the two object waves pass through the micropolarizer array, the polarization directions of the waves are aligned, and the waves interfere with each other. The phase shifts required for phase-shifting interferometry28^,29 are spatially introduced to the interference fringes by utilizing circular polarizations and each transmission axis of each micropolarizer. The phase shift becomes $2\theta$. Therefore, self-interference holograms with phase shifts of 0, 90, 180, and 270 deg at RGB bands are simultaneously recorded with a single-shot exposure. The demosaicking procedure is applied to the single recorded hologram in a computer, and four holograms at each color channel are numerically generated. Phase-shifting interferometry and diffraction integrals are calculated with the numerically generated holograms, and then, the full-color object images focused at arbitrary depths are reconstructed. A motion picture of the holograms is obtained at the frame rate of the image sensor.

We have developed a portable natural-light motion-picture holographic camera to demonstrate the applicability of the proposed holographic method. The camera is composed of a camera lens, polarimetric optical elements, and a color polarization image sensor, as shown in Fig. 2(a). The camera lens minifies and images the 3D information of an object and a scene illuminated by spatially and temporally incoherent light. The minified and imaged 3D information of incoherent light is recorded as an object wave. The polarizer sets the linear polarization for the object wave. The birefringent lenses generate two incoherent object waves whose wavefront curvature radii are different and whose polarization directions are orthogonal to each other. The birefringent plates and variable polarimetric phase modulator adjust the optical-path-length difference between the two waves because the careful adjustment of the difference considering the coherence length is important for temporally incoherent light. The quarter-wave plate converts the two linearly polarized object waves into circular polarizations with opposite handedness. With a color polarization image sensor, a self-interference hologram, $H(x,y;\theta )$, of an object wave is recorded. The mathematical expression of $H(x,y;\theta )$ is as follows: $$\begin{gathered} H(x,y;\theta )=\mathit{CFA}(x,y)·{\mathit{H}}_{\mathrm{RGB}}(x,y;\theta ) \\ =[T_{{\lambda }_R}(x,y),T_{{\lambda }_G}(x,y),T_{{\lambda }_B}(x,y)]·[H_{{\lambda }_R}(x,y;\theta ),H_{{\lambda }_G}(x,y;\theta ),H_{{\lambda }_B}(x,y;\theta )], \end{gathered}$$(1)where $\mathit{CFA}(x,y)$ is the spectral transmission distribution of the color filter array, $T_{\lambda j}(x,y)$ is the transmission distribution at the wavelength band $\mathrm{\Delta }{\lambda }_j$ ($j=R,G,B$), $\lambda$ is the wavelength of light, ${\mathit{H}}_{\mathrm{RGB}}(x,y;\theta )$ is the spectral self-interference hologram, and $H_{\lambda j}$$(x,y;\theta )$ is the self-interference hologram with the wavelength band $\mathrm{\Delta }{\lambda }_j$ and is expressed as $$\begin{gathered} H_{{\lambda }_j}(x,y;\theta )={\int }_{{\lambda }_{cj}-\mathrm{\Delta }{\lambda }_j/2}^{{\lambda }_{cj}+\mathrm{\Delta }{\lambda }_j/2}\iiint I(x,y,{\mathit{r}}_{\mathit{o}},\lambda ;\theta )\mathrm{d}{x}_o\mathrm{d}{y}_o\mathrm{d}{z}_o\mathrm{d}\lambda \\ =H_{0{\mathrm{th}}_{{\lambda }_j}}(x,y)+U_{{\lambda }_j}(x,y)e^{-i(2\theta -\pi /2+\varphi )}+C.C.. \end{gathered}$$(2)

Here, $I(x,y,{\mathit{r}}_{\mathit{o}},\lambda ;\theta )$ is the self-interference hologram of the object point located at ${\mathit{r}}_{\mathit{o}}=(x_o,y_o,z_o)$ in the object and is mathematically expressed as $$\begin{gathered} I(x,y,{\mathit{r}}_{\mathit{o}},\lambda ;\theta )=|C({\mathit{r}}_{\mathit{o}},\lambda )L[\frac{-(x_{\mathit{o}},y_{\mathit{o}})}{z_1}\left]Q\right(\frac{1}{z_1}\left)\right[Q\left(\frac{-1}{f_1}\right)e^{-i(\theta -\pi /4)}*Q\left(\frac{1}{z_2\prime }\right)+Q\left(\frac{-1}{f_2}\right)e^{i(\theta -\pi /4)}{e}^{i\varphi }*Q\left(\frac{1}{{z_2}^{\prime }}\right)]{|}^2 \\ =|C^{\prime }({\mathit{r}}_{\mathit{o}},\lambda )L\left[\frac{-f_a(x_{\mathit{o}},y_{\mathit{o}})}{z_1(f_a+z_2\prime )}\right]Q\left(\frac{1}{f_a+z_2\prime }\right)e^{-i(\theta -\pi /4)}+C^{\prime }({\mathit{r}}_{\mathit{o}},\lambda )L\left[\frac{-f_b(x_{\mathit{o}},y_{\mathit{o}})}{z_1(f_b+z_2\prime )}\right]Q\left(\frac{1}{f_b+{z_2}^{\prime }}\right)e^{i(\theta -\pi /4)}{e}^{i\varphi }{|}^2 \\ =I_{0\mathrm{th}}(x,y;{\mathit{r}}_{\mathit{o}})+C^{‴}({\mathit{r}}_{\mathit{o}},\lambda )L[\frac{-M}{f_c}(x_{\mathit{o}},y_{\mathit{o}})]Q\left(\frac{-1}{f_c}\right)e^{-i(2\theta -\pi /2)}{e}^{-i\varphi }+c.c., \end{gathered}$$(3)where $U_{\lambda j}(x,y)$ is the incoherent object wave with the wavelength band $\mathrm{\Delta }{\lambda }_j$, which contains 3D information of the object and is generated from the incoherent sum of self-interference holograms of multiple object points, and is mathematically expressed as $$U_{{\lambda }_j}(x,y)={\int }_{{\lambda }_{cj}-\mathrm{\Delta }{\lambda }_j/2}^{{\lambda }_{cj}+\mathrm{\Delta }{\lambda }_j/2}\iiint C^{‴}({\mathit{r}}_{\mathit{o}},\lambda )L[\frac{-M}{f_c}(x_{\mathit{o}},y_{\mathit{o}})]Q\left(\frac{-1}{f_c}\right)\mathrm{d}{x}_o\mathrm{d}{y}_o\mathrm{d}{z}_o\mathrm{d}\lambda ,$$(4)where ${\lambda }_{cj}$ is the center wavelength at $\mathrm{\Delta }{\lambda }_j$; $H_{0{\mathrm{th}}_{{\lambda }_j}}(x,y)$ and $I_{0\mathrm{th}}(x,y)$ are the 0th-order diffraction waves of $H(x,y;\theta )$ and $I(x,y,{\mathit{r}}_{\mathbf{o}},\lambda ;\theta )$, respectively; $i$ is the imaginary unit; C.C. and c.c. are the complex conjugates of $U(x,y)$ and the second term of Eq. (3), respectively; $C({\mathit{r}}_{\mathbf{o}},\lambda )$, $C^{\prime }({\mathit{r}}_{\mathbf{o}},\lambda )$, $C^{″}({\mathit{r}}_{\mathbf{o}},\lambda )$, and $C^{‴}({\mathit{r}}_{\mathbf{o}},\lambda )$ are coefficients; $z_1$ is the depth difference between an object point and the closely set birefringent lenses; $L(x_o,y_o)=\exp [i2\pi (x_{o}x+y_{o}y)/\lambda ]$11; $Q(1/z)=\exp [i\pi (x^2+y^2)/\lambda z]$11; * indicates a convolution; $z_2$ is the depth difference between the closely set birefringent lenses and the image sensor; ${z_2}^{\prime }=z_2-d_p/[1-(1/n_f)]$; ${z_2}^{″}=z_2-d_p/[1-(1/n_s)]$; $d_p$ is the total thickness of the closely set birefringent plates; $n_f$ and $n_s$ are the refractive indices of the fast and slow axes of the closely set birefringent plates, respectively; $\varphi$ is the relative phase shift from the object point to the pixel placed on the optical axis, which is induced by the closely set birefringent lenses, variable phase modulator, and closely set birefringent plates; $f_1$ and $f_2$ are the synthesized focal lengths of the closely set birefringent lenses in the fast and slow axes, respectively; $f_a=f_1{z}_1/(f_1-z_1)$; $f_b=f_2{z}_1/(f_2-z_1)$; $M=(f_a{z_2}^{″}-f_b{z_2}^{\prime })/[z_1(f_a+{z_2}^{\prime }-f_b-{z_2}^{″})]$ is the magnification of the IDH system; and $f_c=(f_a+{z_2}^{\prime })(f_b+{z_2}^{″})/(f_a-f_b+{z_2}^{\prime }-{z_2}^{″})$ is the numerical focusing distance. ${z_2}^{\prime }$ and ${z_2}^{″}$ indicate that the focal length shifts slightly owing to the birefringent plate and are derived with paraxial approximation.30 The adjustment of $\varphi$ is a key for improving the visibility of the interference fringes of the temporally incoherent light. Where $d_l$ is the total thickness of the closely set birefringent lenses on the optical axis, the refractive indices of the fast and slow axes of the birefringent lenses are the same as those of the birefringent plates, ${\varphi }_{\mathrm{pm}}$ is the phase shift generated by the birefringent variable phase modulator, and $k_c$ is the central wavenumber, we set the following relationship as a guideline for the adjustment: $$\varphi =(k_c{n}_f{d}_l+k_c{n}_s{d}_p)-(k_c{n}_s{d}_l+k_c{n}_f{d}_p)+{\varphi }_{pm}=0,(\text{when}{d}_l\ge d_p).$$(5)

Static $k_c$ can be set using the color filter array. Equation (5) indicates that ${\varphi }_{\mathrm{pm}}$ is set to be $\varphi =0$, and the incoherent light wave incident from the optical axis is recorded as an incoherent hologram with the zero optical-path-length difference. As a result, interference fringes with high visibility are obtained for the object placed on the optical axis. When the material of the lenses is the same as that of the plates, $\varphi =0$ is obtained by setting $d_l=d_p$ accurately. As another aspect, the visibility of interference fringes generated by an arbitrary wave-vector component is flexibly adjusted by changing ${\varphi }_{\mathrm{pm}}$. The attenuation of interference fringes is related to the shape of the spectrum, the optical-path-length difference, and the spectral bandwidth.30 Equations (2) and (3) indicate that the phase shift of an incoherent hologram depends on $\theta$, which changes pixel by pixel. Where $\varphi =0$ is set, $p_x$ and $p_y$ are the pixel pitches of the image sensor in the $x$- and $y$-axis directions, $x/p_x=-N_x/2,\dots ,0,\dots ,N_x/2-1$ and $y/p_y=-N_y/2,\dots ,0,\dots ,N_y/2-1$ have discrete integer values, which means the address of the photodetector of the image sensor, $N_x$ and $N_y$ are the numbers of pixels in the $x$- and $y$-axis directions, respectively, and $H_{\lambda j}$$(x,y;\theta )$ is expressed as $$H_{{\lambda }_j}(x,y;\theta )=\{\begin{array}{lll}{H}_{{\lambda }_j}(x,y;\pi /4),& \text{where}(\frac{x}{p_x}+\frac{N_x}{2})=\text{even},& (\frac{y}{p_y}+\frac{N_y}{2})=\text{even}\\ H_{{\lambda }_j}(x,y;0),& \text{where}(\frac{x}{p_x}+\frac{N_x}{2})=\text{even},& (\frac{y}{p_y}+\frac{N_y}{2})=\text{odd}\\ H_{{\lambda }_j}(x,y;3\pi /4),& \text{where}(\frac{x}{p_x}+\frac{N_x}{2})=\text{odd},& (\frac{y}{p_y}+\frac{N_y}{2})=\text{odd}\\ H_{{\lambda }_j}(x,y;\pi /2),& \text{where}(\frac{x}{p_x}+\frac{N_x}{2})=\text{odd},& (\frac{y}{p_y}+\frac{N_y}{2})=\text{even}\end{array}\mathrm{.}$$(6)

Here, we assume a simple spectral transmittance distribution of $\mathit{CFA}(x,y)$ as $$\begin{gathered} \mathit{CFA}(x,y)=[T_{{\lambda }_R}(x,y),T_{{\lambda }_G}(x,y),T_{{\lambda }_B}(x,y)] \\ =\{\begin{array}{l}(\mathrm{1,0},0),\text{where}(\frac{x}{p_x}+\frac{N_x}{2})\equiv p(\mathrm{mod}4)\text{and}(\frac{y}{p_y}+\frac{N_y}{2})\equiv p(\mathrm{mod}4)(p=\mathrm{0,1})\\ (\mathrm{0,1},0),\text{where}\left[\right(\frac{x}{p_x}+\frac{N_x}{2})\equiv p(\mathrm{mod}4)\text{and}(\frac{y}{p_y}+\frac{N_y}{2})\equiv q(\mathrm{mod}4)](q=\mathrm{2,3})\\ \text{or}\left[\right(\frac{x}{p_x}+\frac{N_x}{2})\equiv q(\mathrm{mod}4)\text{and}(\frac{y}{p_y}+\frac{N_y}{2})\equiv p(\mathrm{mod}4)]\\ (\mathrm{0,0},1),\text{where}(\frac{x}{p_x}+\frac{N_x}{2})\equiv q(\mathrm{mod}4)\text{and}(\frac{y}{p_y}+\frac{N_y}{2})\equiv q(\mathrm{mod}4)\end{array}\mathrm{.} \end{gathered}$$(7)

Equations (1)–(7) indicate that the information of four phase-shifted incoherent digital holograms at RGB channels is recorded with a single-shot exposure. Figure 2(b) shows a photograph of the camera that we developed. The dimensions of the body (black box) were 104 mm (W) × 108 mm (H) × 204 mm (D). A CANON camera lens (EF 100 mm F2.8L Macro IS USM for the initial and second experiments and EF11-24mm f/4L USM for the last experiment) and a SONY color polarization image sensor (IMX250MYR) were adopted in the developed camera. A crystal was selected as the birefringent material. A THORLABS multi-order liquid crystal phase modulator (LCC2415-VIS) was used as the variable polarimetric phase modulator.

Full-color 3D information of an incoherent object wave is retrieved without noise from a single-recorded hologram by applying the image-reconstruction procedure illustrated in Fig. 3. The recorded hologram is transferred to a computer to extract the object-wave information from the recorded image by signal processing. Multiple-phase-shifted incoherent holograms at each color channel are numerically generated from the recorded hologram by applying the demosaicking procedure.19 This time, an interpolation procedure25 is not applied to the demosaicked holograms. The complex amplitude distribution at each color channel is extracted by calculating the phase-shifting interferometry.28^,29 Incoherent object waves at RGB color channels on the image sensor plane, $U_{{\lambda }_R}(x,y)$, $U_{{\lambda }_G}(x,y)$, and $U_{{\lambda }_B}(x,y)$, are extracted using the following equations of phase-shifting interferometry. $$U_{{\lambda }_R}(x,y)=H_{{\lambda }_R}(4x,4y+1)-H_{{\lambda }_R}(4x+\mathrm{1,4}y)+i[H_{{\lambda }_R}(4x,4y)-H_{{\lambda }_R}(4x+\mathrm{1,4}y+1)],$$(8)$$\begin{gathered} U_{{\lambda }_G}(x,y)=\{H_{{\lambda }_G}(4x,4y+3)-H_{{\lambda }_G}(4x+\mathrm{1,4}y+2)+i[H_{{\lambda }_G}(4x,4y+2)-H_{{\lambda }_G}(4x+\mathrm{1,4}y+3)] \\ +H_{{\lambda }_G}(4x+\mathrm{2,4}y+1)-H_{{\lambda }_G}(4x+\mathrm{3,4}y)+i[H_{{\lambda }_G}(4x+\mathrm{2,4}y)-H_{{\lambda }_G}(4x+\mathrm{3,4}y+1)]\}/2, \end{gathered}$$(9)$$U_{{\lambda }_B}(x,y)=H_{{\lambda }_B}(4x+\mathrm{2,4}y+3)-H_{{\lambda }_B}(4x+\mathrm{3,4}y+2)+i[H_{{\lambda }_B}(4x+\mathrm{2,4}y+2)-H_{{\lambda }_B}(4x+\mathrm{3,4}y+3)]\mathrm{.}$$(10)

Numerical light-wave propagations based on diffraction integrals are calculated for the object waves, and the object images focused on arbitrary depth planes are reconstructed at RGB color channels. Denoising procedures, such as nonlocal means (NLM),31 3D block matching (BM3D),32 and 4D block matching (BM4D)33 filtering and/or machine-learning-based algorithms such as denoising based on MIRNet,34^,35 are applied to the reconstructed images to enhance the image quality. Finally, the full-color 3D information of objects illuminated by spatially and temporally incoherent light is holographically reconstructed without noise.

We have conducted experiments using the portable natural-light motion-picture holographic camera that we developed. Initially, we set the camera and two objects on a movable wooden table with five wheels. The light source was only the sun, and the sunlight scattered by the ground glass of the window was used as illumination light. Therefore, a full-color digital hologram was generated with fully passive and spatially and temporally incoherent light. Two clothes pegs on the table and a shoji screen were set as the objects. The distance between the two clothes pegs was 75 mm and that between the snowman-shaped peg and the shoji was 680 mm. A natural-light full-color digital hologram was recorded by a single-shot exposure with an exposure time of 10 ms. An NLM filter31 was applied to the reconstructed images at different depths as denoising algorithms. For comparison, reconstructed images without denoising were also obtained. Figure 4 shows the experimental results, which indicate that full-color three-object images were successfully reconstructed from a single natural-light hologram. A full-color digital hologram was obtained even with sunlight, which is spatially and temporally incoherent, at a 10-ms exposure time of the polarization color image sensor. Moreover, the image quality was successfully enhanced by applying denoising algorithms that are frequently applied to general incoherent imaging. Then, we investigated the imaging properties of the denoising algorithms and the qualities of the reconstructed images. Figure 5(a) shows the reconstructed image after applying a BM3D32 filter. The image shows a considerably improved quality compared with the images shown in Fig. 4(d), and it was clarified that denoising worked correctly. Figure 5 also shows the difference in the imaging properties of BM3D and NLM. Each denoising method successfully suppressed random noise. Moreover, as indicated by the red arrows in Fig. 5, the applied NLM filter suppressed artifacts due to diffractions from the image sensor plane and different object planes. It is considered that an NLM filter can suppress such artifacts that are frequently seen in digital holography when an object has mainly low-spatial-frequency components. To evaluate the image qualities quantitatively, we calculated standard deviations for constant intensity-value areas, which are indicated by yellow rectangles in Fig. 5(a). Table 1 shows the calculation results. It was quantitatively clarified that the image quality was considerably improved by strongly suppressing random noise with the numerical filters. The noise-suppression powers of the NLM and BM3D filters were at the same level.

In the next experiment, we set the camera facing toward the outside of a room to capture an outdoor scene illuminated by sunlight. A metallic bar and other outdoor objects such as a house, trees, and a blue net were recorded as a full-color digital hologram and reconstructed as focused and defocused images, respectively. The exposure time was 10 ms. A BM3D filter was applied for denoising utilizing sparsity and nonlocal self-similarity in an image. Furthermore, we obtained 40 holograms and reconstructed images and then applied the BM4D filter33 to the reconstructed images. Figure 6 shows the experimental results, which indicate that full-color images of an outdoor scene at different depths are reconstructed from a single natural-light hologram. The image quality was also successfully enhanced by applying denoising algorithms that are frequently applied to general incoherent imaging. Both BM3D and BM4D filters strongly suppressed random noise. For detailed analyses of the reconstructed images, we selected and magnified areas containing the high-spatial-frequency components. Figure 7 shows the magnified areas and plots for high spatial frequency components of the recorded scene. Figures 7(a)–7(f) indicate that BM3D partly sacrificed the resolution when denoising although BM3D suppresses the loss of the resolution in comparison with other denoising algorithms. The loss of the resolution was avoided, and the contrast of high spatial frequency components was improved by applying a BM4D filter. It was clarified that a BM4D filter is useful for denoising in the holographic motion-picture recording of static objects. We calculated standard deviations for constant-intensity-value areas, which are indicated by yellow rectangles in Figs. 6(b) and 6(d), as quantitative evaluations. Table 2 shows the calculation results. It was quantitatively clarified that the image quality was much improved by strongly suppressing random noise with the numerical filters. The noise-suppression powers of BM3D and BM4D filters were at the same level.

We carried out an additional experiment to simultaneously demonstrate the capabilities of recording full-color 3D image information and close-up recording, as well as the applicability of the proposed holography to a moving reflective object. We set two objects, a human subject and a miniature model of a finch, which were placed at different depths, and captured the movements of the human subject as an incoherent motion-picture hologram with the developed camera. Figure 8(a) shows the experimental setup. The holographic camera was set on a movable table without any antivibration structure. The distances from the camera lens to the finch and from the finch to the human subject were 45 mm and 1.2 m, respectively. We used a white LED (RC220D, Shenzen Leqi Network Technology Co., Ltd.) to illuminate the human subject and red, green, and blue LEDs with nominal wavelengths of 625, 530, and 455 nm and full width at half-maximum values of 18, 33, and 18 nm to illuminate a finch, respectively. The frame rate and exposure time per frame were 22 fps and 45 ms, respectively. Denoising based on MIRNet34^,35 was applied to the reconstructed images, and then, an NLM filter31 was applied for further denoising. Finally, the motion picture of the reconstructed images was generated. Figures 8(b)–8(g) and Videos 1 and 2 show the experimental results, which indicate that when the numerical refocusing of respective objects is performed, the focused image of a finch is obtained under the condition of close-up recording, and the motion of the human subject is successfully reconstructed from the recorded holograms. From the results, the capabilities of recording full-color 3D image information and close-up recording, and the applicability of the proposed holography to a moving reflective object were experimentally demonstrated simultaneously. Then, we comparatively and quantitatively evaluated the reconstructed images. Figure 9 shows the reconstructed images with and without denoising algorithms. Figures 9(c) and 9(d) indicate that denoising works for a human face and does not work well for a miniature model of the finch by applying denoising based on MIRNet. It is considered that the dataset used for machine learning36^,37 affected the performance of denoising. Residual random noise on the finch and artifacts on the human face were suppressed by applying an NLM filter. Table 3 shows the calculated standard deviations whose areas are indicated by the yellow rectangles shown in Figs. 9(a) and 9(b). It was quantitatively clarified that the image quality for the human face was much improved by strongly suppressing random noise with the MIRNet-based denoising. The MIRNet-based denoising also suppressed random noise on the finch. An NLM filter suppressed noise in the area around the finch.

We discuss the specifications of the proposed holography such as spatial resolution, light-use efficiency, exposure time, and frame rate. The spatial resolution in 3D space is determined by the bottleneck between the finest pitch of interference fringes and the pixel pitch of demosaicked holograms. The proposed holography obtains the temporal resolution of a color polarization image sensor at the cost of the partial spatial resolution of interference fringes. Because of the exploitation of space-division multiplexing, the pixel pitch of each phase-shifted hologram in the proposed holography is four times that in phase-shifting incoherent digital holography without space-division multiplexing. The resolution of interference fringes is down to 1/4, and thus, the 3D spatial resolution is also principally 1/4. In the application to the microscopy of the proposed holography, diffraction-limited imaging can be conducted using a microscope objective. However, the space-bandwidth product available for recording an object wave is principally limited by applying space-division multiplexing.38 The proposed holography adopts a linear polarizer and a micropolarizer array, and therefore, three-fourths of the light intensity of the incident incoherent object wave is discarded. The maximum light-use efficiency of the proposed holography is 25% in principle, and four times the exposure time is required under the ultimately weak light condition. The frame rate of the proposed holography is determined by that of a color polarization image sensor and light intensity. The light-use efficiency also affects the frame rate and thus the maximum frame rate is limited under the condition of severely weak light intensity per exposure. Therefore, the proposed holography is useful for the conditions where an optical design set in front of the holography system supports the spatial resolution, light intensity is not ultimately weak per exposure, and dynamics is captured by the exposure time and the frame rate of a color polarization image sensor.

We have proposed the method of single-shot natural-light full-color holography in which motion-picture holograms are obtained at the frame rate of a color polarization image sensor. The camera based on the proposed holography is portable and usable on a daily use table. Fully passive full-color holographic 3D imaging was achieved with only sunlight and an exposure time of 10 ms. The full-color holographic 4D imaging of a moving object was performed using spatially and temporally incoherent light. Different from time-of-flight imaging and 3D measurement technologies with structured light such as fringe-projection profilometry, the proposed holography achieves fully passive full-color both single-shot 3D imaging and 3D motion-picture recording at the frame rate of a color polarization image sensor. The focused image is obtained at a large depth, and it is considered that the proposed holography can help to improve the depth-measurement range of conventional 3D measurement technologies such as fringe-projection profilometry and stereo vision by combining them.

We believe that the proposed holography method and the developed camera will open up a new stage toward fully passive full-color 4D holography, holographic motion-picture recording with daily-use illumination light, and full-color video-rate holography outdoors under the sun, which overcomes the limitations of conventional holographic imaging.

Tatsuki Tahara received his DE degree from Kyoto Institute of Technology, Japan, in 2013. He was an assistant professor of Kansai University, Japan, from 2013 to 2018; a specially-appointed associated professor of National Institute of Informatics, Japan, from 2018 to 2019; a researcher with PRESTO, JST, Japan, from 2016 to 2020; and a researcher of National Institute of Information and Communications Technology, Japan, from 2019 to 2021, where he has been a senior researcher, since 2021.

Tomoyoshi Shimobaba received his BE and ME degrees from Gunma University, Japan in 1997 and 1999, respectively. He received his DE degree from Chiba University, Japan, in 2002. From 2002 to 2005, he was a special postdoctoral researcher at RIKEN. From 2005 to 2009, he was an associate professor at the Graduate School of Science and Engineering, Yamagata University, Japan. From 2009 to 2019, he was an associate professor in the Graduate School of Engineering, Chiba University, Japan. He is currently a professor in the Graduate School of Engineering, Chiba University, Japan.

Yuichi Kozawa received his PhD from Tohoku University, Japan, in 2008. He joined the Institute of Multidisciplinary Research for Advanced Materials (IMRAM), Tohoku University, Japan, as an assistant professor in 2008. In 2016, he worked as an associate professor; since 2024, he has been a professor of IMRAM, Japan. He is a member of Optica, the Japan Society of Applied Physics (JSAP), the Optical Society of Japan (OSJ), and the Laser Society of Japan (LSJ).

Mohamad Ammar Alsherfawi Aljazaerly received his BS degree in information technology engineering from Damascus University, Syria, in 2016, and the MS degree in computer science from Osaka University, Japan, in 2020, where he is currently pursuing his PhD in computer science. His research interests include computer vision, image processing, deep learning, and gait recognition.

Tomoya Nakamura is an associate professor at SANKEN, Osaka University, Japan. He received his PhD from Osaka University, Japan, in 2015. From 2015 to 2020, he served as an assistant professor at Tokyo Institute of Technology, Japan. His research interests include computational imaging and holography. He is a member of Optica.