• Advanced Photonics
  • Vol. 1, Issue 1, 016004 (2019)
Zhenbo Ren1、2, Zhimin Xu3, and Edmund Y. Lam1、*
Author Affiliations
  • 1University of Hong Kong, Department of Electrical and Electronic Engineering, Pokfulam, Hong Kong, China
  • 2Northwestern Polytechnical University, School of Natural and Applied Sciences, Xi’an, China
  • 3SharpSight Limited, Hong Kong, China
  • show less
    DOI: 10.1117/1.AP.1.1.016004 Cite this Article Set citation alerts
    Zhenbo Ren, Zhimin Xu, Edmund Y. Lam. End-to-end deep learning framework for digital holographic reconstruction[J]. Advanced Photonics, 2019, 1(1): 016004 Copy Citation Text show less
    (a) Schematic of the deep learning workflow and the structure of HRNet. It consists of three functional blocks: input, feature extraction, and reconstruction. In the first block, the input is a hologram of either an amplitude object (top), a phase object (middle), or a two-sectional object (bottom). The third block is the reconstructed output image according to the specific input. The second block shows the structure of HRNet; (b) and (c) elaborate the detailed structures of the residual unit and the subpixel convolutional layer, respectively.
    Fig. 1. (a) Schematic of the deep learning workflow and the structure of HRNet. It consists of three functional blocks: input, feature extraction, and reconstruction. In the first block, the input is a hologram of either an amplitude object (top), a phase object (middle), or a two-sectional object (bottom). The third block is the reconstructed output image according to the specific input. The second block shows the structure of HRNet; (b) and (c) elaborate the detailed structures of the residual unit and the subpixel convolutional layer, respectively.
    (a) The USAF test target and its local areas as amplitude objects. (b) A customized groove on an optical wafer as the phase object. (c) A homemade two-sectional object consisting of a transparent triangle and a rectangle located at different axial positions.
    Fig. 2. (a) The USAF test target and its local areas as amplitude objects. (b) A customized groove on an optical wafer as the phase object. (c) A homemade two-sectional object consisting of a transparent triangle and a rectangle located at different axial positions.
    Experimentally collected testing holograms of amplitude objects.
    Fig. 3. Experimentally collected testing holograms of amplitude objects.
    (a)–(d) Ground-truth images and reconstructed images of holograms in Fig. 3 using (e)–(h) HRNet, (i)–(l) ASM, and (m)–(p) CONV.
    Fig. 4. (a)–(d) Ground-truth images and reconstructed images of holograms in Fig. 3 using (e)–(h) HRNet, (i)–(l) ASM, and (m)–(p) CONV.
    Experimentally collected testing holograms of the phase object.
    Fig. 5. Experimentally collected testing holograms of the phase object.
    (a)–(d) Ground-truth images and reconstructed quantitative phase images of holograms in Fig. 5 using (e)–(h) HRNet, (i)–(l) PCA, and (m)–(p) DE. The unit of the color bar is radian.
    Fig. 6. (a)–(d) Ground-truth images and reconstructed quantitative phase images of holograms in Fig. 5 using (e)–(h) HRNet, (i)–(l) PCA, and (m)–(p) DE. The unit of the color bar is radian.
    Experimentally collected testing holograms of the two-sectional object.
    Fig. 7. Experimentally collected testing holograms of the two-sectional object.
    Ground-truth: (a)–(d) EFI and (e)–(h) DM. HRNet, reconstructed: (i)–(l) EFI and (m)–(p) DM. Entropy, reconstructed: (q)–(t) EFI and (u)–(x) DM. T-gradient, reconstructed: (y)–(ab) EFI and (ac)–(af) DM. Variance, reconstructed: (ag)–(aj) EFI and (ak)–(an) DM. The color bar shows the depth in DM; the unit is mm.
    Fig. 8. Ground-truth: (a)–(d) EFI and (e)–(h) DM. HRNet, reconstructed: (i)–(l) EFI and (m)–(p) DM. Entropy, reconstructed: (q)–(t) EFI and (u)–(x) DM. T-gradient, reconstructed: (y)–(ab) EFI and (ac)–(af) DM. Variance, reconstructed: (ag)–(aj) EFI and (ak)–(an) DM. The color bar shows the depth in DM; the unit is mm.
    (a) and (b) Holograms. (c) and (d) Frequency spectra. (e) and (f) Reconstructed images under different angles.
    Fig. 9. (a) and (b) Holograms. (c) and (d) Frequency spectra. (e) and (f) Reconstructed images under different angles.
    Holograms [(a) and (b)] and reconstructed images [(c) and (d)] under different axial distances.
    Fig. 10. Holograms [(a) and (b)] and reconstructed images [(c) and (d)] under different axial distances.
    MeasureMethodsAmplitude dataset
    ValidationTest
    PSNR (dB)ASM17.6619.64
    CONV19.6820.54
    HRNet25.9924.62
    SSIMASM0.200.19
    CONV0.260.26
    HRNet0.920.91
    Time (s)ASM1.561.49
    CONV1.351.72
    HRNet1.141.21
    Table 1. Comparison of reconstruction performance for the amplitude object among ASM, CONV, and HRNet.
    MeasureMethodsPhase dataset
    ValidationTest
    PSNR (dB)PCA10.129.53
    DE8.948.68
    HRNet30.3530.49
    SSIMPCA0.130.11
    DE0.120.10
    HRNet0.960.96
    Time (s)PCA1.961.93
    DE2.092.15
    HRNet1.061.20
    Table 2. Comparison of reconstruction performance for the phase object among PCA, DE, and HRNet.
    MeasureMethodsEFIDM
    ValidationTestValidationTest
    PSNR (dB)SEN16.8215.9212.6612.78
    VAR15.4414.6912.7811.92
    TEN16.0315.8611.8212.24
    HRNet35.6435.7237.8136.70
    SSIMSEN0.280.270.800.80
    VAR0.100.110.820.82
    TEN0.140.100.800.80
    HRNet0.970.970.970.98
    Time (s)SEN380.30392.68390.03391.36
    VAR384.38386.52390.58388.82
    TEN376.76383.66398.29394.37
    HRNet1.351.301.041.42
    Table 3. Comparison of EFI and DM reconstruction performance for the two-sectional object among SEN, VRA, TEN, and HRNet.
    Layer numberLayer typeConfigurationNumber of parameters
    Layer 12-D convolution3 × 3 × 32 + BN + ReLU3 × 3 × 32 = 288
    Layer 2ResUnit (64)Max-pooling: 2 × 2 3 × 3 × 64 + BN + ReLU 3 × 3 × 64 + BN + ReLUParameter-free 3 × 3 × 32 × 64 = 18,432 3 × 3 × 64 × 64 = 36,864
    Layer 3ResUnit (64)3 × 3 × 64 + BN + ReLU 3 × 3 × 64 + BN + ReLU3 × 3 × 64 × 64 = 36,864 3 × 3 × 64 × 64 = 36,864
    Layer 4ResUnit (128)Max-pooling: 2 × 2 3 × 3 × 128 + BN + ReLU 3 × 3 × 128 + BN + ReLUParameter-free 3 × 3 × 64 × 128 = 73,728 3 × 3 × 128 × 128 = 147,456
    Layer 5ResUnit (128)3 × 3 × 128 + BN + ReLU 3 × 3 × 128 + BN + ReLU3 × 3 × 128 × 128 = 147,456 3 × 3 × 128 × 128 = 147,456
    Layer 6ResUnit (256)Max-pooling: 2 × 2 3 × 3 × 256 + BN + ReLU 3 × 3 × 256 + BN + ReLUParameter-free 3 × 3 × 128 × 256 = 294,912 3 × 3 × 256 × 256 = 589,824
    Layer 7ResUnit (256)3 × 3 × 256 + BN + ReLU 3 × 3 × 256 + BN + ReLU3 × 3 × 256 × 256 = 589,824 3 × 3 × 256 × 256 = 589,824
    Layer 8Subpixel convolution3 × 3 × 64 + BN + ReLU + periodic shuffling3 × 3 × 256 × 64 = 147,456
    Total parameters2,857,248
    Table 4. Detailed description of the layers and parameters of the proposed HRNet (biases are ignored in the computation).
    Zhenbo Ren, Zhimin Xu, Edmund Y. Lam. End-to-end deep learning framework for digital holographic reconstruction[J]. Advanced Photonics, 2019, 1(1): 016004
    Download Citation