• Laser & Optoelectronics Progress
  • Vol. 58, Issue 14, 1400001 (2021)
Shaohui Zhang, Guocheng Zhou, Baiqi Cui, Yao Hu, and Qun Hao*
Author Affiliations
  • School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
  • show less
    DOI: 10.3788/LOP202158.1400001 Cite this Article Set citation alerts
    Shaohui Zhang, Guocheng Zhou, Baiqi Cui, Yao Hu, Qun Hao. Review of Fourier Ptychographic Microscopy: Models, Algorithms, and Systems[J]. Laser & Optoelectronics Progress, 2021, 58(14): 1400001 Copy Citation Text show less
    Optical imaging principle and system setup of FPM. (a) Imaging principle of FPM[26]; (b) system setup of FPM[37]; (c) constraints in the Fourier domain; (d) illumination of oblique LEDs; (e) LEDs are sequentially turned up during FPM image acquisition
    Fig. 1. Optical imaging principle and system setup of FPM. (a) Imaging principle of FPM[26]; (b) system setup of FPM[37]; (c) constraints in the Fourier domain; (d) illumination of oblique LEDs; (e) LEDs are sequentially turned up during FPM image acquisition
    Schematic diagrams of the range of diffracted light collected by the objective lens at different illumination light angles[41]. (a) Under normal incident illumination; (b) under oblique incident illumination
    Fig. 2. Schematic diagrams of the range of diffracted light collected by the objective lens at different illumination light angles[41]. (a) Under normal incident illumination; (b) under oblique incident illumination
    Constraints and algorithm flow of alternating projection[21]. (a) Constraint with support domain; (b) constraint with amplitude; (c) algorithm flow of alternating projection
    Fig. 3. Constraints and algorithm flow of alternating projection[21]. (a) Constraint with support domain; (b) constraint with amplitude; (c) algorithm flow of alternating projection
    Phase recovery flow chart of Fourier ptychography microscopy[26]
    Fig. 4. Phase recovery flow chart of Fourier ptychography microscopy[26]
    Iterative update flow chart of FPM imaging[44]. (a) Sequential update; (b) batch update
    Fig. 5. Iterative update flow chart of FPM imaging[44]. (a) Sequential update; (b) batch update
    LED array positional misalignment correction method based on hardware[56]. (a) System setup; (b1)(b2) criterion for roll angle adjustment of the LED array; (c1)~(c4) criterion for adjustment of central LED position; (d) a mosaic image combining raw images of aperture diaphragm containing bright-field components
    Fig. 6. LED array positional misalignment correction method based on hardware[56]. (a) System setup; (b1)(b2) criterion for roll angle adjustment of the LED array; (c1)~(c4) criterion for adjustment of central LED position; (d) a mosaic image combining raw images of aperture diaphragm containing bright-field components
    LED array positional misalignment model of FPM setup[57]. (a) Diagram of a misaligned FPM setup; (b) enlargement of the central windowed part in Fig. 7(a)
    Fig. 7. LED array positional misalignment model of FPM setup[57]. (a) Diagram of a misaligned FPM setup; (b) enlargement of the central windowed part in Fig. 7(a)
    Results and system setup of efficient illumination angle self-calibration FPM[58]. (a) Brightfield images contain overlapping circles in their Fourier spectra, while darkfield images do not; (b) system setup with quasi-dome type illuminator
    Fig. 8. Results and system setup of efficient illumination angle self-calibration FPM[58]. (a) Brightfield images contain overlapping circles in their Fourier spectra, while darkfield images do not; (b) system setup with quasi-dome type illuminator
    Illumination patterns with planar and dome type LED arrays[60]. (a) Planar type LED array; (b) dome type LED array
    Fig. 9. Illumination patterns with planar and dome type LED arrays[60]. (a) Planar type LED array; (b) dome type LED array
    Dome type LED illumination. (a)(b) Designed by Phillips et al[60]; (c)--(e) designed by Pan et al[62]
    Fig. 10. Dome type LED illumination. (a)(b) Designed by Phillips et al[60]; (c)--(e) designed by Pan et al[62]
    Schematic diagram of intensity distribution along different angles of LED[41]
    Fig. 11. Schematic diagram of intensity distribution along different angles of LED[41]
    Algorithm flow of full-field FPM algorithm based on gradient descent method[65]
    Fig. 12. Algorithm flow of full-field FPM algorithm based on gradient descent method[65]
    Reconstruction results from uncorrected FPM and EPRY-FPM algorithms [64]. (a1)(a2) Reconstructed sample amplitude and phase using uncorrected FPM algorithm; (b1)(b2) reconstructed sample amplitude and phase using EPRY-FPM algorithm; (c1)(c2) reconstructed pupil function modulus and phase using EPRY-FPM algorithm
    Fig. 13. Reconstruction results from uncorrected FPM and EPRY-FPM algorithms [64]. (a1)(a2) Reconstructed sample amplitude and phase using uncorrected FPM algorithm; (b1)(b2) reconstructed sample amplitude and phase using EPRY-FPM algorithm; (c1)(c2) reconstructed pupil function modulus and phase using EPRY-FPM algorithm
    Reconstruction results based on full-field FPM [65]. (a) Reconstructed full-FOV amplitude with full-field FPM; (b1)(c1)(d1) reconstructed amplitudes corresponding to three positions; (b2)(c2)(d2) reconstructed phase corresponding to three positions; (b3)(c3)(d3) raw images corresponding to three positions
    Fig. 14. Reconstruction results based on full-field FPM [65]. (a) Reconstructed full-FOV amplitude with full-field FPM; (b1)(c1)(d1) reconstructed amplitudes corresponding to three positions; (b2)(c2)(d2) reconstructed phase corresponding to three positions; (b3)(c3)(d3) raw images corresponding to three positions
    FPM system constructions with laser illumination. (a)(b) FPM imaging setup and the Fourier spectrum distribution based on 2D Galvo mirror system proposed by Chung et al[67]; (c) FPM imaging setup based on DMD proposed by Kuang et al[68]; (d)(e) FPM imaging setup based on LCD proposed by Guo et al[69]
    Fig. 15. FPM system constructions with laser illumination. (a)(b) FPM imaging setup and the Fourier spectrum distribution based on 2D Galvo mirror system proposed by Chung et al[67]; (c) FPM imaging setup based on DMD proposed by Kuang et al[68]; (d)(e) FPM imaging setup based on LCD proposed by Guo et al[69]
    Schematic diagrams of light collection angle range of objective lens[38]. (a) Critical angles of bright-field and dark-field images; (b) illumination angle that corresponds to boundary of bright-field and dark-field images; (c) diagram of bright-field and dark-field images
    Fig. 16. Schematic diagrams of light collection angle range of objective lens[38]. (a) Critical angles of bright-field and dark-field images; (b) illumination angle that corresponds to boundary of bright-field and dark-field images; (c) diagram of bright-field and dark-field images
    FPM reconstructed results with different denoising methods[72]. (a)(b) Reconstructed result with no denoising and part of the enlarged image; (c)(d) reconstructed result with conventional thresholding denoising and part of the enlarged image; (e)(f) reconstructed result with adaptive denoising and part of the enlarged image
    Fig. 17. FPM reconstructed results with different denoising methods[72]. (a)(b) Reconstructed result with no denoising and part of the enlarged image; (c)(d) reconstructed result with conventional thresholding denoising and part of the enlarged image; (e)(f) reconstructed result with adaptive denoising and part of the enlarged image
    Iteration equations weighting factor and penalty function factor corresponding to varied probe amplitude for different algorithms[78]. (a) Iteration equations weighting factor; (b) penalty function factor
    Fig. 18. Iteration equations weighting factor and penalty function factor corresponding to varied probe amplitude for different algorithms[78]. (a) Iteration equations weighting factor; (b) penalty function factor
    Typical probe patterns in PIE/FPM[44]. (a) Convergent beam probe commonly used in X ray; (b) random speckle probe commonly used in visible light imaging; (c) ideal circular lowpass filter probe used in FPM
    Fig. 19. Typical probe patterns in PIE/FPM[44]. (a) Convergent beam probe commonly used in X ray; (b) random speckle probe commonly used in visible light imaging; (c) ideal circular lowpass filter probe used in FPM
    Results comparison of adaptive step-size, fixed step-size and some other FPM algorithms[47]. (a) Comparison of convergence speed and reconstruction accuracy of the adaptive step-size approach and four state-of-the-art FPM reconstruction algorithms under 50% Poisson noise; (b) comparison of reconstruction results and runtime of the adaptive step-size approach and four state-of-the-art FPM reconstruction algorithms
    Fig. 20. Results comparison of adaptive step-size, fixed step-size and some other FPM algorithms[47]. (a) Comparison of convergence speed and reconstruction accuracy of the adaptive step-size approach and four state-of-the-art FPM reconstruction algorithms under 50% Poisson noise; (b) comparison of reconstruction results and runtime of the adaptive step-size approach and four state-of-the-art FPM reconstruction algorithms
    Comparison of experimental results between AcFPM and FPM[83]. (a1)(a2) Schematic of traditional FPM and AcFPM; (b1)--(e1) reconstructed amplitude and the corresponding enlarged results of traditional FPM; (b2)--(e2) reconstructed amplitude and the corresponding enlarged results of AcFPM
    Fig. 21. Comparison of experimental results between AcFPM and FPM[83]. (a1)(a2) Schematic of traditional FPM and AcFPM; (b1)--(e1) reconstructed amplitude and the corresponding enlarged results of traditional FPM; (b2)--(e2) reconstructed amplitude and the corresponding enlarged results of AcFPM
    Diagram of neural network structure used to replace the traditional FPM phase recovery algorithm[87]
    Fig. 22. Diagram of neural network structure used to replace the traditional FPM phase recovery algorithm[87]
    Physical-guided neural network of FPM[86]
    Fig. 23. Physical-guided neural network of FPM[86]
    Different sample models for FPM. (a) 2D thin sample; (b) 3D thin sample; (c) 3D thick sample
    Fig. 24. Different sample models for FPM. (a) 2D thin sample; (b) 3D thin sample; (c) 3D thick sample
    Digital refocusing results of FPM introduced with phase propagator and convergence index[95]. (a1)--(a3) FPM reconstructions without digital refocusing; (b1)--(b3) FPM reconstructions with digital refocusing; (c) FPM convergence index as a function of defocused distances
    Fig. 25. Digital refocusing results of FPM introduced with phase propagator and convergence index[95]. (a1)--(a3) FPM reconstructions without digital refocusing; (b1)--(b3) FPM reconstructions with digital refocusing; (c) FPM convergence index as a function of defocused distances
    Algorithm flow of digital refocusing FPM inserted phase propagator in the reconstruction iteration[26]
    Fig. 26. Algorithm flow of digital refocusing FPM inserted phase propagator in the reconstruction iteration[26]
    USAF chart reconstruction results of digital refocusing FPM based on geometric characteristic of the imaging system[97]
    Fig. 27. USAF chart reconstruction results of digital refocusing FPM based on geometric characteristic of the imaging system[97]
    Paramecium sample reconstruction results of digital refocusing FPM based on geometric characteristic of the imaging system[97]. (a1)--(a4) Reconstructed amplitude and phase of conventional and digital refocusing FPM with the defocus distance of 79 μm; (b1)--(b4) reconstructed amplitude and phase of conventional and digital refocusing FPM with the defocus distance of 80 μm; (c1)--(c4) reconstructed amplitude and phase of conventional and digital refocusing FPM with the defocus distance of 60 μm
    Fig. 28. Paramecium sample reconstruction results of digital refocusing FPM based on geometric characteristic of the imaging system[97]. (a1)--(a4) Reconstructed amplitude and phase of conventional and digital refocusing FPM with the defocus distance of 79 μm; (b1)--(b4) reconstructed amplitude and phase of conventional and digital refocusing FPM with the defocus distance of 80 μm; (c1)--(c4) reconstructed amplitude and phase of conventional and digital refocusing FPM with the defocus distance of 60 μm
    3D FPM system setup and reconstruction results based on Multi-Slice model[99]. (a) FPM system setup; (b) 3D reconstruction results
    Fig. 29. 3D FPM system setup and reconstruction results based on Multi-Slice model[99]. (a) FPM system setup; (b) 3D reconstruction results
    Diagrams of sub-spectrum distributions in 2D and 3D FPM. (a) FPM system setup[102]; (b) 2D spectrum andthe corresponding sub-spectrum distribution[102]; (c) 3D spectrum and the corresponding sub-spectrum distribution[101]
    Fig. 30. Diagrams of sub-spectrum distributions in 2D and 3D FPM. (a) FPM system setup[102]; (b) 2D spectrum andthe corresponding sub-spectrum distribution[102]; (c) 3D spectrum and the corresponding sub-spectrum distribution[101]
    Algorithm flow of 3D FPM and results of tomography based on phase recovery and sub-spectrum splicing in the 3D Fourier domain[101]. (a) Algorithm flow of 3D FPM; (b) reconstructed results of tomography 3D FPM
    Fig. 31. Algorithm flow of 3D FPM and results of tomography based on phase recovery and sub-spectrum splicing in the 3D Fourier domain[101]. (a) Algorithm flow of 3D FPM; (b) reconstructed results of tomography 3D FPM
    HDR FPM reconstructed results[104]. (a) Raw image of a blood smear sample; (b) reconstructed result without HDR combination process; (c) reconstructed result with HDR combination process; (d) reconstructed result with sparsely sampled FPM
    Fig. 32. HDR FPM reconstructed results[104]. (a) Raw image of a blood smear sample; (b) reconstructed result without HDR combination process; (c) reconstructed result with HDR combination process; (d) reconstructed result with sparsely sampled FPM
    Principle and reconstructed results of adaptive HDR FPM with RGB camera[105]. (a)(b) Imaging principle of Bayer filter; (c) system setup of adaptive HDR FPM; (d) large FOV LR images acquired by this system; (e1)--(e3) reconstructed results with single channel of RGB; (f) reconstructed result with adaptive HDR FPM
    Fig. 33. Principle and reconstructed results of adaptive HDR FPM with RGB camera[105]. (a)(b) Imaging principle of Bayer filter; (c) system setup of adaptive HDR FPM; (d) large FOV LR images acquired by this system; (e1)--(e3) reconstructed results with single channel of RGB; (f) reconstructed result with adaptive HDR FPM
    Diagrams of different illumination patterns[106]. (a) FPM with single LED pattern; (b) FPM with LED positions multiplexing pattern; (c) FPM with wavelengths multiplexing pattern
    Fig. 34. Diagrams of different illumination patterns[106]. (a) FPM with single LED pattern; (b) FPM with LED positions multiplexing pattern; (c) FPM with wavelengths multiplexing pattern
    FPM algorithm flow of spectral multiplexing[106]. (a) LED positions multiplexing; (b) LED wavelengths multiplexing
    Fig. 35. FPM algorithm flow of spectral multiplexing[106]. (a) LED positions multiplexing; (b) LED wavelengths multiplexing
    FPM reconstructed results of coded illumination by multiple LEDs[73]. (a) Low resolution image acquired by camera; (b) a zoom-in on the smallest features; (c) reconstruction result from traditional FPM; (d) multiplexing FPM with 293 images; (e) multiplexing FPM with 74 images
    Fig. 36. FPM reconstructed results of coded illumination by multiple LEDs[73]. (a) Low resolution image acquired by camera; (b) a zoom-in on the smallest features; (c) reconstruction result from traditional FPM; (d) multiplexing FPM with 293 images; (e) multiplexing FPM with 74 images
    FPM reconstructed results corresponding to different overlapping of sub-spectrum[104]。 (a1)(a2) Input intensity and phase with high resolution in the simulation; (b)--(d) FPM reconstructions with different spectrum overlapping percentages in the Fourier domain
    Fig. 37. FPM reconstructed results corresponding to different overlapping of sub-spectrum[104]。 (a1)(a2) Input intensity and phase with high resolution in the simulation; (b)--(d) FPM reconstructions with different spectrum overlapping percentages in the Fourier domain
    Updated LED illumination arrays of FPM[108]. (a) System setup of FPM; (b)(c) different perspectives of illuminators
    Fig. 38. Updated LED illumination arrays of FPM[108]. (a) System setup of FPM; (b)(c) different perspectives of illuminators
    Updated FPM system setups. (a)(b) FPM system setup based on a cellphone lens[112]; (c) FPM system setup based on Raspberry Pi and the corresponding open sources[113]; (d) FPM system setup based on multi-aperture camera array[114]; (e) FPM system setup based on industrial camera and telecentric objective[56]
    Fig. 39. Updated FPM system setups. (a)(b) FPM system setup based on a cellphone lens[112]; (c) FPM system setup based on Raspberry Pi and the corresponding open sources[113]; (d) FPM system setup based on multi-aperture camera array[114]; (e) FPM system setup based on industrial camera and telecentric objective[56]
    Imaging principle of lens-less FPM imaging. (a) System setup of lens-less FPM proposed by Luo et al[116]; (b) system setup of lens-less FPM proposed by Zhang et al[117]
    Fig. 40. Imaging principle of lens-less FPM imaging. (a) System setup of lens-less FPM proposed by Luo et al[116]; (b) system setup of lens-less FPM proposed by Zhang et al[117]
    System setup of FPM with reflection and epi-illumination sources. (a) System setup of FPM based on epi-illumination[121]; (b) system setup of FPM based on reflection illumination designed by Guo et al[119]; (c) system setup of FPM based on a parabolic mirror[123]
    Fig. 41. System setup of FPM with reflection and epi-illumination sources. (a) System setup of FPM based on epi-illumination[121]; (b) system setup of FPM based on reflection illumination designed by Guo et al[119]; (c) system setup of FPM based on a parabolic mirror[123]
    Principle and system setup of macroscopic FP imaging. (a) Reflection geometry FP imaging[125]; (b)(c) aperture-scanning FP imaging[126]
    Fig. 42. Principle and system setup of macroscopic FP imaging. (a) Reflection geometry FP imaging[125]; (b)(c) aperture-scanning FP imaging[126]
    AlgorithmPenalty function factor μjrWeighting factor wjrUpdated function of object
    PIEPjrmaxPjr+αPjrmax2Pjr-Pjr2Pjr3Pjrmax(Pjr2+αPjrmax2)O'jr=Ojr+PjrPjr*(ψ'jr-ψjr)Pjrmax(Pjr2+αPjrmax2)
    ePIE1αPjrmax2-Pjr2αPjr2Pjrmax2O'jr=Ojr+αPjr*(ψ'jr-ψjr)Pjrmax2
    rPIEα(Pjrmax2-Pjr2)Pjr2(1-α)Pjr2+αPjrmax2O'jr=Ojr+Pjr*(ψ'jr-ψjr)(1-α)Pjr2+αPjrmax2
    Table 1. Object update functions for PIE, ePIE and rPIE algorithms
    Shaohui Zhang, Guocheng Zhou, Baiqi Cui, Yao Hu, Qun Hao. Review of Fourier Ptychographic Microscopy: Models, Algorithms, and Systems[J]. Laser & Optoelectronics Progress, 2021, 58(14): 1400001
    Download Citation