Fig. 1. Brief overview of the structure of the review.
Fig. 2. Achromatic computational imaging based on metasurface modulation techniques. EDoF-based achromatic imaging: (a) cubic phase mask, reproduced with permission from Ref.
30 (CC-BY), (b) symmetric EDoF-based canonical phase mask, reproduced with permission from Ref.
31 © 2020 Chinese Laser Press. Optimization-based inverse design methods: (c) multizone dispersion-engineered metalens-based achromatic RGB focusing, reproduced with permission from Ref.
32 (CC-BY), (d) achromatic RGB imaging with efficient 3D inverse design methods, reproduced with permission from Ref.
33 (CC-BY), (e) achromatic visible imaging with EDoF-based inverse design, reproduced with permission from Ref.
31 © 2021 American Chemical Society, and (f) achromatic inverse design based upon artificial neural network, reproduced with permission from Ref.
35 © 2021 Wiley-VCH.
Fig. 3. Spectral modulation-based hyperspectral imaging. Hyperspectral imaging with random encoding with (a) regular-shaped metasurface design, reproduced with permission from Ref.
2 © 2022 Optical Society of America, (b) freeform-shaped metasurface, reproduced with permission from Ref.
44 © 2022 Wiley-VCH, (c) metasurface-based gratings, reproduced with permission from Ref.
45 © 2022 Optica Publishing Group, (d) multi-aperture spectral filter array-based hyperspectral imaging, reproduced with permission from Ref.
46 (CC-BY), (e) end-to-end learned optimal metasurface design, reproduced with permission from Ref.
3 © 2022 IEEE.
Fig. 4. Polarization modulation-based computational metasurface imager. (a) Polarization multiplexing-based single-pixel imaging, reproduced with permission from Ref.
55 (CC-BY), (b) extreme-DoF imaging with polarization multiplexing, reproduced with permission from Ref.
59 (CC-BY), (c), (d) wide FoV microscopic imaging methods with polarization multiplexing, reproduced with permission from Ref.
58 (CC-BY) and Ref.
57 (CC-BY), (e) polarization multiplexing for underwater descattering, reproduced with permission from Ref.
56 © 2021 Wiley-VCH, (f) polarization multiplexing-based 4D imaging, reproduced with permission from Ref.
60 (CC-BY), (g) full-Stokes imaging, reproduced with permission from Ref.
4 © 2019 AAAS, (h) efficient polarization imaging with polarization splitting, reproduced with permission from Ref.
61 © 2020 Optical Society of America, (i) compressive polarization imaging with random weak dichroism metasurface, reproduced with permission from Ref.
5 (CC-BY).
Fig. 5. Depth modulation and imaging techniques with metasurface imager. (a) DH-PSF engineering-based depth imaging, reproduced with permission from Ref.
65 (CC-BY), (b) EDoF and DH-based PSF engineering, with side-by-side metalens, reproduced with permission from Ref.
66 © 2020 ACS, (c) depth from dual-defocus multiplexing, inspired by jumping spider vision, reproduced with permission from Ref.
67 (CC-BY), and (d) triple metalens based 3D positioning, reproduced with permission from Ref.
68 © 2020 Optica Publishing Group.
Fig. 6. Angle dimension modulation for computational imaging. Wide-angle-imaging-based upon: (a) ommatidia-inspired pixel-wise angle-sensitive filtering, reproduced with permission from Ref.
70 (CC-BY), (b) angle-selective metalens array, reproduced with permission from Ref.
71 © 2022 Optica Publishing Group, (c) synthetic aperture with four small apertures, reproduced with permission from Ref.
72 © 2021 Chinese Laser Press. Wide-angle illumination for 3D depth imaging based upon: (d) pseudo-random coding, reproduced with permission from Ref.
73 (CC-BY), (e) uniform dense light patterns, reproduced with permission from Ref.
6 (CC-BY), (f) double-zone illumination, reproduced with permission from Ref.
74 (CC-BY), and (g) dual depth imaging mode with structured light-field imaging under common light conditions and structured imaging under low-light conditions, reproduced with permission from Ref.
75 © 2022 Wiley-VCH.
Fig. 7. Spectrum-angle/depth modulation-based computational imaging: spectral-angle joint modulation for (a) RGB and depth imaging with EDoF, reproduced with permission from Ref.
89 © 2021 ACS, (b) spectral light-field imaging, reproduced with permission from Ref.
91 (CC-BY), (c), (d) high-efficiency color imaging based upon color routing, reproduced with permission from Ref.
93 (CC-BY) and Ref.
92 © 2021 ACS, (e) single-image multichannel imaging, reproduced with permission from Ref.
94 © 2022 Optica Publishing Group, and (f) full-color wide-FoV imaging, reproduced with permission from Ref.
95 (CC-BY).
Fig. 8. Computational imaging framework of metasurface-based imaging. (a) The computational imaging process, containing optimizable imaging component and reconstruction algorithms; (b) independent optimization framework; and (c) end-to-end optimization framework.
Fig. 9. Metasurface modulation-based computational sensing methods.
| Illumination | Sensing | Reconstruction | Spectrum | | • Canonical EDoF phase-based achromatic design30,31 | • Wiener filtering36 | • Inversely-designed achromatic encoding34 | • TV-regularized convex optimization algorithm97,102,103 | • Random hyperspectral coding with regularly-shaped,2 free-form shaped,44 grating-based metasurface45 | • MLP-based spectral reconstruction3 | • Multiaperture hyperspectral filtering46 | • Dictionary learning-based sparse recovery48 | • PCA-based spectral encoding3 | • End-to-end learned metasurface design and neural network based hyperspectral retrieval and segmentation3,51 | Polarization | | • Trilobite-inspired chiral multiplexing59 | • Multiscale CNN59 | • Linear polarization multiplexing60 | • Matrix inversion4 | • Inversely-designed polarization splitter with tetrahedron polarization analyzer4 | • Gradient descent optimization algorithm23 | • Metalens array-based polarization splitting61 | • Deep mask-aware compressive full-Stokes reconstruction network5 | • Random polarization filtering5 | • Mosaic polarization filtering63 | Depth/Wide angle | • High-density 1D or 2D dots () or parallel lines (), covering 180 deg FoV6 | • Synthetic aperture sensing72 | • Truncated SVD technique76 | • M-array-based pseudo-random coding for illumination73 | • Ommatidia-inspired pixel-wise planar wide-angle selective sensing70 | • Wiener filtering36 and the Richardson–Lucy deconvolution algorithm77,78 | • Double-zone illumination, i.e., peripheral zone (150 deg FoV) and center zone (2 deg FoV)74 | • Angle-selective metalens linear array-based wide-angle sensing71 | • Mask-based stitching algorithm71 | • Depth extraction algorithm based on triangulation,87 stereo-matching,88 and time-of-flight74 | PSF engineering | | • DH-PSF65,66,69 | • Cepstrum analysis65 | • Jumping spider-inspired dual-defocus PSF67 | • Depth from defocus algorithm104 | • Three-subaperture based on three close-packed hexagonal metalenses68 | • Depth from parallex of correlation-aberration corrected subaperture images68 | Compound | | • Dispersion-based spectral encoding and metalens array-based light-field imaging91 | • Convex optimization of 4D hyperspectral light-field image based on spectral–spatial sparsity prior97 | • Inversely-design-based color routing or splitting92,93 | • U-Net-based RGB and depth retrieval neural network89 | • Tri-focus PSF over RGB channels89 | • End-to-end learned metasurface design and wide FoV imaging95 and snapshot multichannel (spectrum, polarization, and depth) imaging94 |
|
Table 1. Overall summarization of existing computational metasurface imager.