• Acta Photonica Sinica
  • Vol. 51, Issue 7, 0751408 (2022)
An PAN1、2、*, Yuting GAO1、2, Aiye WANG1、2, Huiqin GAO1、2, Caiwen MA1、2, and Baoli YAO1、2
Author Affiliations
  • 1Xi'an Institute of Optics and Precision Mechanics,Chinese Academy of Sciences,Xi'an 710119,China
  • 2University of Chinese Academy of Sciences,Beijing 100094,China
  • show less
    DOI: 10.3788/gzxb20225107.0751408 Cite this Article
    An PAN, Yuting GAO, Aiye WANG, Huiqin GAO, Caiwen MA, Baoli YAO. High-throughput Full-color Fourier Ptychographic Microscopy for the Next Generation of Digital Pathologic Imager and Analyser(Invited)[J]. Acta Photonica Sinica, 2022, 51(7): 0751408 Copy Citation Text show less

    Abstract

    Fourier Ptychographic Microscopy (FPM) is a promising computational imaging technique, which tackles the intrinsic trade-off between high resolution and large wide Field Of View (FOV) with a combination of Synthetic Aperture Radar (SAR) and optical phase retrieval. In brief, an LED array beneath the microscope provides illumination of the object from different incident angles. The range of light that can be collected is determined by the Numerical Aperture (NA) of the objective, while parts of the scattering light with a high-angle illumination can also be collected because of light-matter interaction. The low resolution intensity images recorded at each illumination angle are then synthesized in the Fourier domain, thus the object’s high-frequency information can be modulated into the passband of the objective. After an iterative phase reconstruction process, the synthesized information generates a high resolution object image including both intensity and phase properties. Additionally, it preserves the original large FOV as a low-NA objective is used to stitch low resolution images together. Given its flexible setup without mechanical scanning and interferometric measurement, FPM has developed rapidly, which not only acts as a tool to obtain both HR and large FOV but is also regarded as a paradigm to solve a series of trade-off problems, say, the trade-off between angular resolution and spatial resolution in light field imaging. And it may inspire to solve the trade-off between spectral resolution and spatial resolution in imaging spectrometer in the future.In this paper, we comprehensively summarized the development trend of FPM technique in 9 aspects, including high-precision imaging, high-throughput imaging, high-speed or single shot imaging, 3D or tomography imaging, mixed state decoupling, spectral dimension (color imaging to hyperspectral imaging), high dynamic range, system extension, and typical applications. Among them, digital pathology is one of the earliest and the most successful applications of FPM. Distinguished from other reviews, we focused on introducing the development process and recent advances in the direction of digital pathology, and divided it into “0-1”, “1-10”, and “10-100” three periods and several stages. Several typical results are also provided. Specifically, the “0-1” refers to the birth of FPM, which breaks the mutual restrictions between FOV and spatial resolution. The “1-10” refers to the exploration period, where the accuracy and stability, limits and bottlenecks, and the efficiency of FPM have been successively discussed and improved. The stage of “10-100” refers to the industrialization period. During this period, researchers focus on market-oriented requirements including acquisition and analysis of color, since full-color imaging is of critical importance for analyzing labeled tissue sections.We point out that FPM has entered the industrialization stage of “10-100” in this application direction.The current task is to build a prototype or product based on FPM. We expect that the product can obtain a spatial resolution of around 200 nm~1 000 nm, a FOV of around 10 mm (2× objective) or 5 mm (4× objective) diameter full-color FPM reconstructed image within 4 s at the DOF of around 0.3~0.5 mm stably and efficiently. We estimate that it can be capable of automation and batch scanning within the next 1~2 years. We analyzed the industry development situations of digital pathology and related market requirements, and discussed the potential of FPM for large-scale socio-economic benefits. We demonstrated that the full-color images with high quality and content and quantitative phase images produced by FPM may play a role of promotion in wide fields, including intraoperative pathology, quantitative Artificial Intelligence (AI) diagnosis, three-dimensional reconstruction, telepathology, teaching and standardized industry criteria. It should also be clarified that as a typical interdisciplinary field, even if the instrument is successfully invented, it only solves issues in the imaging section of the whole process of digital pathology, and there still remain a series of tough tasks to complete. We discussed and classified related scientific problems, technical problems, engineering problems, and industrial problems in detail, whose successful and perfect resolution relies on joint efforts of various parties and constructive introduction of several potential approaches. By combining the FPM solution with the upstream and downstream advanced methods, including the virtual staining, multimodal fusion imaging, label-free observation in situ, non-destructive three-dimensional reconstruction, preliminary screening, and recognition with AI, etc., we believe that the industry problems will eventually be overcome or alleviated.

    0 Introduction

    Pathology is a bridge between basic research and clinical applications. Analyzing pathology slides using an optical microscope remains as the gold standard in diagnosing a large number of diseases including almost all types of cancers. Compared with the department of clinical laboratory and picture archiving in a typical hospital of China as shown in Table 1,the pathologic diagnosis can obtain definitive diagnosis,but it requires 5~10 workdays to get the final reports. Note that intraoperative pathology is quite sensitive to time. Given the fact that,on the one hand,the pathology department has higher requirements for personal experience and takes a long time to train pathologists. And currently,there is a relative shortage of pathologists worldwide. On the other hand,it has less automatic and digital equipment,which causes that the diagnostic procedures are divided into multiple steps that are conducted by different pathologists and also the department is labor-intensive. These difficulties synthetically result in the long-time diagnosis.

    DepartmentsChecklistRequired timeType of diagnosis
    Clinical laboratoryBiochemistry & ImmunityHepatic and renal function2 hoursDeductive Diagnosis
    Blood glucose,lipids,electrolytes
    Routine immunization
    LuminescenceTumor markersOn that day
    Thyroid function
    Gonadal hormone
    BacteriaSmear and stainingOn that day
    Bacterial culture4 hours
    Picture archivingX-ray1 hours
    Ultrasound
    Magnetic Resonance Imaging(MRI)
    PathologyConventional small sample3 daysDefinitive Diagnosis
    Large samples of surgical removal5 days
    Immunohistochemistry,molecular pathology7~10 days

    Table 1. The information of checklist and its required time at a typical hospital in China

    Specifically,traditional pathology has four main steps:sample preparation,imaging,analysis,and report output. At the heart of these steps is the imaging step,where the imaging quality directly affects the subsequent judgments. Currently,traditional pathology still relies on observation through a microscope eyepiece with the naked eyes. It is necessary to switch back and forth between objective lens with different magnification and shift Field Of View(FOV)to detect pathological features. The operation may be easily disrupted,and the features may be easily omitted. It is also highly subjective as different pathologists may arrive at different conclusions or the same pathologist may give different conclusions at different time points. For a piece of tissue with hundreds to thousands of slices,undoubtedly,there is a heavy workload and would be low efficiency1-2.

    With the rapid development of computational power and the digital trend of picture achieving,communication and telediagnosis,modern digital pathology adopts digital imaging and mechanical scanning to obtain High Resolution(HR)first and then large FOV so as to achieve full FOV imaging of a single slice,termed Whole Slide Imaging(WSI)system1-2. This kind of instrument is the so-called digital pathology scanner,while it still has to overcome four intrinsic defects. First,the imaging quality and success rate is not high enough:due to the drift of electronic devices,the stitching boundary is prone to artifacts(Fig. 1(a)). To overcome it,an elaborate registration or feature matching algorithm is required. However,for sparse samples,due to a large amount of blank area and lack of adjacent features,they may not be spliced(Fig. 1(b)),and the feature matching errors tend to cause double image(Fig. 1(c)). As a result,the success rate is directly related to the kind of samples and is relatively low due to those artifacts. Second,the application scenarios are limited:Due to the narrow Depth Of Field(DOF)of a high Numerical Aperture(NA)objective,it is limited to histopathology or hematology and cannot be applied in cytopathology for a certain thickness(Fig. 1(d)). Even it may also fail due to air bubbles. Thus,axial scanning may be needed for conventional WSI system,adding another degree of complexity of data handling to the system. Third,the efficiency is relatively low:Among the four-step workflow of translation,autofocusing,imaging and stitching,the translation and autofocusing need to be performed repeatedly at high speed,while the imaging time accounts for a small proportion and the detector is idling for most of the time. Fourth,the instrument is expensive:Precise mechanical control and pulsed illumination are often needed to shorten the acquisition time. The resultant systems are expensive and of high maintenance. Considering all the issues above,digital pathology is not universally accepted by pathologists currently and is only regarded as a supplementary tool in limited fields like research,teleconsultation and etc.

    The intrinsic drawbacks of conventional digital pathologic scanners

    Figure 1.The intrinsic drawbacks of conventional digital pathologic scanners

    Fourier Ptychographic Microscopy(FPM)3-10 was invented in 2013 by introducing the concept of ptychography11-14 into the reciprocal(Fourier)space,which breaks the trade-off between large FOV and HR with a combination of synthetic aperture radar15-16 and optical phase retrieval17-18. On the hardware side,the condenser of conventional optical microscopy is replaced by a cheap LED array,which is still compatible with modern microscopes. And the LED elements are lighted up sequentially to achieve the synthetic aperture and resolution improvement. In theory,the objective can only collect light ranging from a certain angle,characterized by the NA. However,parts of the scattering light with a high-angle illumination can also be collected because of light-matter interaction. The sample's high-frequency information can be modulated into the passband of the objective lens. Instead of conventionally stitching small HR image tiles into a large FOV,FPM uses a low NA objective innately with a large FOV to stitch together Low Resolution(LR)images in Fourier space and finally obtain HR images. Compared with conventional digital pathology,firstly,no mechanical scanning is required for the angle-scanning scheme,therefore,the stitching artifacts,double images,splicing failure problems are all solved fundamentally,and the image quality is guaranteed. Secondly,the detector would not be idling and keeps working,therefore,the FPM solution is a high-throughput technique(~1 gigapixels). Thirdly,the low NA objective has an innate long DOF and working distance,and the DOF can also be digitally extended to more than 300μm3 according to the principle of coherent imaging. Note that the slicer can cut the sample up to 100 μm,which is fully covered by the DOF so that it is not easy to lose focus. Fourthly,the aberration can be digitally recovered and compensated. Fifthly,additional quantitative phase information can be provided and used for quantitative biology. In a nutshell,the FPM solution ideally solves the above-mentioned problems of conventional digital pathology and offers some unique advantages that are not available previously.

    Currently,FPM has been written into the book of the Introduction to Fourier Optics(4th edition)by Goodman19. Given its flexible setup without mechanical scanning and interferometric measurement,FPM has improved rapidly,which not only acts as a tool to obtain both HR and large FOV but is also regarded as a paradigm to solve a series of trade-off problems,e.g.,the trade-off between angular resolution and spatial resolution in light field imaging20. And it may inspire to solve the trade-off between spectral resolution and spatial resolution of the imaging spectrometer in the future. The trend of FPM on several aspects is illustrated in Fig. 2 including high-precision imaging21-38,high-throughput imaging39-43,high-speed and single shot imaging44-58,three-dimensional(3D)imaging and tomography59-64,mixed-state decoupling65-71,wavelength diversity72-77,high dynamic range imaging78,system extension79-101 and several typical applications in label-free Quantitative Phase Imaging(QPI)102-107,cell culture and drug screening108-109,Circulating Tumor Cell(CTC)analysis110,retina imaging111,wafer detection112,optical cryptosystem113-114,and remote sensing16115-116,etc. Among these 9 aspects,the first 5 aspects and parts of the applications can be referred to those comprehensive reviews of Refs.[89]in detail. The wavelength,dynamic range and system extension are new trends. Some results indicate that the wavelength multiplexing FPM achieves the spectral resolution of around 2 nm and develops towards hyperspectral imaging77. The FPM algorithm can also extend the dynamic range of single photon avalanche diode array78. In terms of the system extension,the light source can be changed to laser array or laser and 2D Galvo mirror79-80,LED and 2D Galvo mirror81,and liquid crystal display82,etc. The reflective construction offers possibilities for optical remote sensing16116 and optical detection112. Benefit from the aberration removal of FPM,the single lens design makes the optical system more compact. And the aperture scanning scheme completely changes the thinking set of multi-angle scanning scheme in FPM and provides the abilities of DOF extension83-85 and incoherent FPM111. Combined with the speckle illumination,the FPM can broaden the scene of applications and achieve the scattering imaging and multimodal imaging.

    The trend of FPM on several aspects

    Figure 2.The trend of FPM on several aspects

    In this paper,we focused on reviewing the application of FPM in digital pathology,since the digital pathology is the initial application area of FPM,and it may be one of the most successful applications of FPM currently and develops rapidly. The origin and basic principle of FPM are introduced in Section 1 in brief. We emphatically review its multiple stages and significant advances toward the next generation of digital pathologic imager and analyzer in Section 2 and point out that it has entered the industrialization stage of “10-100” in this direction of digital pathology. In Section 3,we discuss its potential for large-scale economic benefits and its impacts on both upstream and downstream ecology. The high-quality imaging results of FPM will significantly benefit the subsequent judgement and analysis,and will provide high-quality data for auxiliary diagnosis via Artificial Intelligence(AI). Finally,we summarize our topic and put forward some unsatisfactory points at present and give prospects for the future in Section 4.

    1 Origin and principle

    The name of FPM is taken from ptychography. Similar to the terminology of holography,“holo” comes from “oλo” in Greek and means “all”,namely the hologram contains all the wavefront information including both amplitude and phase. “Ptycho” comes from the Greek word “πτυξ” that means “to fold”117,because the diffraction pattern is a convolution between the Fourier transform of a finite illumination function and the Fourier spectrum of the object,which is illustrated in Fig. 3(a). At the beginning,ptychography was proposed in 1969 to solve the phase problem in electron diffraction measurement with only one intensity image and is one kind of the methods of Coherent Diffraction Imaging(CDI)117. However,the phase retrieval with a single diffraction pattern is sensitive to the noise and is difficult to converge or fall into local minimum18. With the development of light sources,detectors,computational power and advanced algorithms35118-128,FAULKNER H M L et al. in 200411118 modified it to record multiple diffraction patterns through the scan of an overlapped localized illumination on an extended object,which extended its capabilities to image non-crystal objects and also applied it to X-ray129-140 and optical imaging setups70141-144. The overlapping regions of adjacent illuminated spots are a coherent superposition,which locks the phase relationship at different positions of the sample and objectively plays the role of the reference beam in the holography that thus no reference arm is required practically8. This set of diffraction patterns is then used to invert the diffraction process and retrieve the complex sample profile. Compared with the hypostasis of vector addition in interferometry,the keystone of diffraction methods relies on vector convolution.

    The comparisons between ptychography[102],FPM[127],and Structured Illumination Microscopy(SIM)

    Figure 3.The comparisons between ptychography102,FPM127,and Structured Illumination Microscopy(SIM)

    Before the introduction of ptychography,CDI and Scanning Transmission X-ray Microscopy(STXM)evolved quite independently and achieved good results in their own field140. The major drawback of CDI is the requirement of crystal or quasicrystal samples,which makes the sample preparation hard and limits its applications. While in STXM,the problem is that the spatial resolution is limited by the size of Fresnel zone plate,which can hardly exceed 10 nm due to the fabrication issues. Ptychography with brilliant X-rays emerged in the late 2 000 s as a method that finally enabled a marriage between the advantages of STXM and CDI,and it overcomes the biggest disadvantages of both techniques140.

    In the case of ptychography as shown in Fig. 3(a),the object support constraints for phase retrieval are imposed by the confined illumination beam in the spatial domain. The reciprocal support constraints are those diffraction patterns in the frequency domain. As such,the sample or probe must be mechanically scanned using the desired FOV. With FPM as shown in Fig. 3(b),the object support constraints are given by the confined Coherent Transfer Function(CTF)in the Fourier domain and the reciprocal support constraints are those LR images in the spatial domain. In this regard,FPM acts as the Fourier counterpart to ptychography,justifying its name. In fact,the LR images of FPM are equivalent to the 90o rotation of ptychographic diffraction patterns in the phase space of 4D Wigner distribution function145,which means that a linear transformation mathematically links these two datasets. And the concept of FPM can also trace its roots back to the relationship between the reciprocal system established in 1994 by LANDAUER M N for electron microscopy146 and the Imaging Interferometric Microscopy(IIM)established by SCHWARZ C J et al. in 2003147-148. Still,the methods only succeeded in experimenting about two decades later because of the development of light sources,detectors,computational power and advanced algorithms. Thus,such computational imaging techniques are also gaining increasing attention8-9149-152.

    In a typical implementation(Fig. 3(b)),FPM employs a programmable LED array for illuminating the sample from different incident angles,contributing to relative displacement between the spectrum of the sample and the CTF due to the Fourier translation theorem19. A low-NA objective lens is used to acquire the corresponding images. In the reconstruction process,the captured LR intensity images are synthesized in the Fourier space to expand the available bandwidth. The synthesized information is then transformed back to the spatial domain to generate an HR object image containing both intensity and phase properties. By using a low-NA objective lens,FPM retains the large FOV set by the lens. With the aperture synthesizing process,it can generate HR images. As such,it achieves both HR and large FOV at the same time. Compared with conventional scanning WSI systems,no mechanical scanning is needed in FPM. The use of a low NA objective also has the benefit of both a long DOF and a long working distance,thereby addressing the above-mentioned challenges of regular WSI systems in Fig. 1.

    Numerically,for each LEDm,n(row m,column n)and its illumination wave vector(kx,m,nky,m,n),under the assumption that the incident light is an ideal plane wave and the sample is relatively thin,the transmitted wave field from the sample oxy)can be described as ox,ye(jxkx,m,n,jyky,m,n),where o is the sample's transmission function,(xy)denotes the 2D Cartesian coordinates in the sample plane,j is the imaginary unit. Then the transmitted light field is Fourier transformed to the Fourier plane and multiplied by the pupil function accordingly,which can be represented as ox,yejxkx,m,n,jyky,m,nPkx,ky,where Pkxky)is the pupil function,which is the coordinate transformation of pupil function pxy)in the spatial domain,termed CTF,and acts as a low-pass filter of an imaging system,(kxky)denotes the 2D spatial frequency coordinates in the Fourier plane,and F denotes the Fourier transform operator defined by

    [o]kx,ky=ox,y=ox,ye-jxkx+ykydxdy

    where the original 2π in the Fourier transform is contained in the 2D spatial frequency coordinates. Afterward,the light field is inverse Fourier transformed to the image plane and the imaging sensor captures an LR intensity image Im,n of the sample,which is given by

    Im,nx,y=-1ox,yejxkx,m,n,jyky,m,nPkx,ky2=-1Okx-kx,m,n,ky-ky,m,nPkx,ky2

    where F -1 is the inverse Fourier transform operator,and O is the Fourier spectrum of the object. The incident wave vector(kx,m,nky,m,n)can be expressed as

    kx,m,n=2πλxm,n-x0xm,n-x02+ym,n-y02+h2ky,m,n=2πλym,n-y0xm,n-x02+ym,n-y02+h2

    where(x0y0)is the central position of each tile after block processing for parallel computation,(xm,nym,n)denotes the position of the LEDm,nλ is the central wavelength,h is the distance between the LED array and sample. Note that the Eq.(3) is not uniform and is related to the specific coordinate system. FPM attempts to eliminate or minimize the variations in amplitude between simulation patterns and captured images iteratively by formulating the following non-convex optimization issue.

    minOkx,kyε=minOkx,kym,nx,yIm,nx,y--1Okx-kx,m,n,ky-ky,m,nPkx,ky2

    To solve this issue,a typical framework of iterative algorithm is given in Fig. 4 that includes 5 steps. 1)Initialize the HR image with interpolation of a LR image. Note that the initial guess can also be random guess and full-one guess. The closer the initial guess is to the truth,the faster the algorithm converges. 2)Generate a LR estimate. 3)Replace the amplitude of LR estimate by the LR intensity measurement. 4)Repeat steps 2-3 for m×n plane wave incidences,then one iteration is completed. 5)Repeat steps 2-4 several times to make the algorithm converge. It is commendable that the FPM framework gracefully solves three significant issues simultaneously:phase recovery,synthetic aperture,and upsampling,while a normal algorithm only solves one thing.

    The flowchart of FPM reconstruction algorithm

    Figure 4.The flowchart of FPM reconstruction algorithm

    So far,many classical and widely used algorithms have been reported,including EPRY-FPM22,Gaussian-Newton25,Wirtinger flow26,mPIE35 and ADMM38,etc.,which basically adopts this iterative framework. The main differences are that their objective function introduces different regularization terms or adopts the L-1 norm compared with Eq.(4),etc. There is a detailed summary in the Refs.[838]and will not be expanded here.

    FPM also shares the root with structured Illumination Microscopy(SIM)due to Euler's formula as shown in Fig. 3(c)and is given as follows. The illumination wavefront can have the same effect with a Cosine structured illumination pattern if the two symmetrical light beams and a normal incident beam illuminate the sample simultaneously

    I01+ejksinθ1x+e-jksinθ1x=I01+2cosksinθ1x

    where θ1 is a certain incident angle. Since three beam information is mixed together in SIM,it is usually appropriate to capture three phase-shift images and recover the HR image along an axis153-155. While in FPM,these three images have already been separated. Three significant differences between FPM and SIM are that firstly SIM can also be used for fluorescence(incoherent)imaging,while the fluorophores are not sensitive to the angle of beams. Secondly,SIM cannot recover the CTF or the point spread function as FPM. Thirdly,FPM can usually improve the resolution over two times,while linear SIM generally improves twice better resolution. Therefore,considering their close connection,the FPM algorithm was also extended to speckle SIM to overcome the resolution limit both with coherent and incoherent light and achieve multimodal fusion imaging with aberration removal85-8997-98101111.

    2 Roadmap on Fourier ptychographic microscopy in digital pathology

    Digital pathology is one of the earliest and most successful applications to adopt FPM technique. As shown in Fig. 5,the development of FPM in digital pathology can be divided into three periods:“0-1”,“1-10”,and “10-100”,and each period may also be divided into several stages. We make it clear that the time nodes in Fig. 5 refers to the initial time of international peers starting this stage. The “0-1” refers to the birth of FPM,which breaks the mutual restrictions between FOV and spatial resolution. The “1-10” refers to the exploration period,where the accuracy and stability,limits and bottlenecks,and the efficiency of FPM have been successively discussed and improved. The stage of “10-100” refers to the industrialization period. During this period,researchers focus on market-oriented requirements including acquisition and analysis of color,since full-color imaging is of critical importance for analyzing labeled tissue sections. And FPM is finally promoted to pathology market by automatic batch scanning and production. The key words and several typical research are listed below the time nodes.

    Roadmap on FPM in digital pathology[3,33,43,57,76,105]

    Figure 5.Roadmap on FPM in digital pathology333435776105

    2.1 First stage:fumbling

    At the beginning of FPM development,the solutions to technical blind spots and systematic flaws were high on the agenda. Therefore,the first stage(around 2014~2018)focused on how to obtain stable and high-accuracy FPM reconstruction results. During this stage,plenty of systematic errors were discovered to rise from stray light32,aberrations2237,dynamic range323,periodic LED-array-induced grid raster24109,LED intensity fluctuation2142-43,LED positional misalignment2530,noise2225-2831-3234-3538121125,vignetting effect36,partial coherence36,mixed errors33 and motion blur29. We have summarized these systematic errors and their corresponding solutions in Table 2 of Ref.[8]. Thus a generalized model can be given as follows.

    Im,nx,y=wm,n-1Okx-kx,m,n-Δkx,m,n,ky-ky,m,n-Δky,m,ne(jkxΔx,jkyΔy)Pkx,kyejϕ2+IN,m,n

    where the weight factor wm,n is introduced by the LED intensity fluctuation,the phase termej is introduced by the aberrations,which is contained in the CTF,the intensity term IN,m,nis introduced by the noise and can be suppressed by a robust algorithm and data preprocessing methods concurrently,the offset term(∆kx,m,n,∆ky,m,n)results from the LED positional misalignment,the phase term e(jkxΔx,jkyΔy) denotes the error of motion blur.

    2.2 Second stage:exploration

    FPM technology makes a great leap in both stability and imaging accuracy after the removal of systematic errors. Several typical results are provided in Fig. 6. Fig. 6(a)shows the results of human adenocarcinoma of breast section. Its FOV is fully covered by a 2× objective,which has a diameter of around 10 mm and is 100 times larger than the FOV of 20× objective,therefore,it does not require to shift the sample. During the second stage(around 2018~2019),researchers attempted to improve the throughput of FPM and seek for its limit. Note that the throughput contains three dimensions:spatial resolution,FOV,and temporal resolution. In this part,some landmark works in throughput enhancement will be introduced. Traditional FPM systems use flat LED panels as light sources,which imposes limitations on the improvement of imaging throughput from two aspects:1)For FPM systems,a larger value of synthetic NA indicates higher spatial resolution. Limited by the planar geometry,the illumination NA cannot arrive at 1,which fails to meet the requirements of digital pathology;2)The actual illumination intensity of marginal LEDs significantly decays compared with the central LED due to the increase of illumination angle43,resulting in a limited range of imaging NA. The achievable illumination NA with the LED panels is around 0.6 tested by a 4×/0.1 NA objective43.

    Several typical FPM results[3,43,56,106]

    Figure 6.Several typical FPM results34356106

    There are mainly two ideas to make a breakthrough in imaging throughput. The first is to increase the NA of objective. For instance,OU X et al.39 obtained a synthetic NA of 1.75 using a 40×/0.75 NA objective in 2015. However,this method achieved an improved resolution with the sacrifice of a large area of FOV,making FPM lose its original elegance. The other idea is to increase the illumination NA of the light source by transforming the design or structure of FPM system. SUN J et al.42 combined the oil-immersion objective with a denser LED array and realized the final synthetic NA of 1.6 based on a 10×/0.4 NA objective. However,due to the limitation of sampling rate in the spatial domain,this method is not applicable to objective with lower magnification. Therefore,the loss of imaging FOV remains noticeable in this method. Inspired by domed LED array40-41,we designed and fabricated a Subwavelength Resolution FPM(SRFPM)43 system based on a hemispherical digital condenser. The system design and reconstruction results are shown in the Fig. 6(b). The SRFPM system uses a 4×/0.1NA objective and achieves an illumination NA of 0.95 under the incident wavelength of 465 nm. Consequently,the limitation of traditional flat LED arrays has been broken,and by calculation the system enables the ultimate performance with a synthetic NA of 1.05,a resolution of 244 nm(group 11,element 1),a FOV of the diameter of 5.5 mm,and a Spatial Bandwidth Product(SBP)of 245 million pixels.

    In addition,for thick samples,only parts of the sample within the DOF may be clearly observed. Extending the DOF and autofocusing are also the tasks of this stage. Benefiting from the properties of coherent light,under the 2×/0.08 NA objective,FPM can perform digital refocusing to extend the DOF to 300 μm,which means that under this condition,FPM can achieve high-throughput imaging with a relatively thick sample. In 2019,SONG P et al.85 added a diffuser between the sample and the objective lens and modulated the outgoing wavefront of the object by shifting the diffuser. Since the recovered image of this method depends on how the complex wavefront leaves the sample rather than entering it,the sample thickness becomes irrelevant during the reconstruction process. The application of this method extends the DOF of FPM from 300 μm to more than 500 μm as shown in Fig. 6(c). Besides,it will be quite challenge for autofocusing if the samples are unstained or weakly-stained,since the contrast will not be high enough as the stained samples. There are multiple methods and can be referred to Refs.[106156-157].

    2.3 Third stage:breakthrough

    In FPM implementation,the HR reconstruction image is obtained depending on hundreds of LR intensity images collected under different angles of illumination along with iterative reconstruction algorithms. Despite its high spatial resolution,the temporal resolution or efficiency of FPM is relatively low. Therefore,improving imaging efficiency has become a research priority for stage 3(around 2019~2021). During this period,several methods have been discussed and they can generally be classified into three types. The first type56103-104 is to reduce the number of images that required to be collected. Based on the assumption that the sample satisfies weak absorption approximation(usually applicable for biological samples),TIAN L et al.103 proposed Differential Phase Contrast(DPC),which requires only four images with asymmetric illumination. The typical result of label-free time-lapse imaging for 5 hours observation of the Hela cells is provided in Fig. 6(d)via the kind of DPC method. The DPC method is undoubtedly fast,but the resolution is much worse than that of FPM. An additional indirect strategy to reduce the image number is to utilize or reduce the information redundancy,specifically including LED multiplexing46-47,Deep Learning(DL)49-50,sparse LED72 and lossy compression algorithms44-45.

    The second type is actually the extreme case for the first type of methods,that is,reducing the number of images to only one shot. The single-shot FPM setup was initially established by HE X and LEE W et al.52-53 based on a micro-lens array. However,the limited area of image sensor leads to a great loss of imaging FOV. After that,several DPC-based color-coding methods53-55 were proposed,where the number of images required can be reduced by half. The disadvantage is that it takes extra time to balance resolution difference,since the absorption is considered identical for different wavelengths. In 2019,we reported a single-shot FPM method(termed sFPM)based on monochromatic illumination with a single defocus image recorded57. This method avoids the operation of resolution balance but is only applicable to sparse samples54. In order to obtain a stable phase for general samples,it is desirable to record two defocus images.

    The third type50 is to complete the reconstruction in the form of “one-step”,which is based on DL. These techniques can be viewed as a supervised learning method that involves potentially large datasets. Other DL-based techniques include a straightforward strategy to replace the reconstruction algorithms with a Convolutional Neural Network(CNN). By adding regularization terms according to certain prior information,this method tends to provide better robustness to noise and some experimental errors. Additionally,DL methods can be combined with the previous two types of methods in order to decrease the computation time or find the optimal illumination patterns.

    2.4 Fourth stage:practical use

    The previous three stages have basically completed the “1-10” period of FPM in digital pathology. In the fourth stage(around 2020~2021),the main task of research is to meet the practical application needs in the field of digital pathology. Here,we discuss the practical use in this stage from the following three aspects:

    1) FPM Colorization

    Since humans are more sensitive to color information and have established the habit of classification according to color,scientists usually stain pathological sections before observation to facilitate specific identification. The combination of colorization and FPM provides further convenience for the detection and identification of samples on the basis of improved imaging throughput. One classical colorization method1473 is that an HR image is restored with a monochrome camera at tri-wavelengths and then synthesized to form one HR full-color image. This process is easy to realize with a programmable R/G/B LED array. However,the reconstruction time is three times that of the original FPM with a single wavelength,and the intensity of different wavelengths must be calibrated. DONG S et al.67 proposed a Wavelength-Multiplexed FPM(WMFPM)scheme with a monochrome camera and multi-wavelength simultaneous illuminations. In this method,the acquisition time is reduced by 2/3,but around three times higher overlapping rate is required. Inspired by the concept of color matching,we reported a high-throughput colorization method of FPM based on color transfer,termed CFPM76. Numerous experiments verified that compared with the traditional R/G/B colorized FPM,CFPM scheme only sacrifices about 0.4% of the imaging precision,which can be ignored,but gains the efficiency by 3-fold.

    2) Multimodal and fluorescence imaging

    Usually before observing biological tissues with a microscope,cells must be dehydrated,coated with toxic dyes or embedded with resin,and sliced. This series of operations may lead to the decline or even loss of cell activity. In the field of digital pathology,the observation and tracking of living cells is of great significance to the screening of tumor cells and the observation of cell morphology. Fluorescence imaging is favored by biologists for its powerful detection ability,slight stimulation to cells and applicability to multiple staining. In 2016,CHUNG J et al.101 combined the fluorescence image after aberration correction and the phase image reconstructed by FPM,which greatly improved the contrast of the original fluorescence image. As shown in Fig. 7(a),we can clearly observe the internal structure of the cells in detail,and further judge the division state of two adjacent cells according to the HR fluorescence phase image.

    Multimodal imaging[101,158]

    Figure 7.Multimodal imaging101158

    3) Polarization FPM

    The research history of polarization is not long because human eyes cannot directly record polarization signal. However,the importance of polarization information cannot be underestimated. Polarization information carries a large amount of structural information of biological tissues and does not need to use exogenous markers,hence has been more and more widely used in biomedical research and clinical practice. The combination of FPM and polarization technology enables a more comprehensive observation and acquisition of detailed cell structures. In 2021,DAI X et al.158 implemented polarization FPM based on Jones matrix,and its measurement strategy and reconstruction results are respectively shown in Fig. 7(b)and Figs. 7(c)~(e). We can simply find that the aberration caused by polarization is effectively corrected. Although there have not been so many reports,the great potential of polarization FPM in biomedical field makes it another research hotspot.

    2.5 Fifth stage:prototype and product

    Currently,FPM has basically satisfied the needs of multiple applications in digital pathology,and its development has entered the 5th stage(around 2021~2022). At this stage,the objective is to invent a digital pathology imager and analyzer with high throughput,high stability,full automation,batch scanning,and low cost,and then promote it to industry by automatic batch production. An example of highly automated and integrated system for pathology analysis is shown in Fig. 5,which is designed by us. We expect to develop a prototype and be capable of mass scanning within the next 1~2 years.

    3 Discussions

    3.1 Market requirement and professional issues

    1) Industry scale

    Analyzing pathology slides using an optical microscope remains as the gold standard in diagnosing a large number of diseases including almost all types of cancers. According to the research report released by Grand View Research159,the global market size of digital pathology in 2021 was valued at USD 926.9 million,and this number is expected to Expand At A Compound Annual Growth Rate(CAGR)of 7.5% from 2022 to 2030. In emerging developing countries especially China,with the trend of industry transformation and upgrading,the development of high-throughput digital pathological scanning instruments leads to faster growth of the demand for high-end microscopes. It is predicted that the potential market of pathology industry in China exceeds 30 billion160,including histopathology(2~3 billion),cytopathology(over 20 billion,especially the screening for cervical cancer),immunohistochemical pathology(over 4 billion),and molecular pathology(over 5 billion).

    2) Industry problems

    Currently,there is an acute shortage of pathologists worldwide. As an example,the average number of pathologists per 100 beds in level-Ⅲ hospitals of China is only 0.3,and many level-Ⅰ hospitals do not even fit up with a pathology department. The main reason is that the occupation has high requirements for personnel quality and professional competence,resulting in a long training cycle. Besides,traditional digital pathology contains a four-step workflow,which is labor-intensive and time-consuming in each section as shown in Fig. 8.

    The potential requirements in the whole workflow of digital pathology to accelerate the output of reports[62,76,109,167-168,172]

    Figure 8.The potential requirements in the whole workflow of digital pathology to accelerate the output of reports6276109167-168172

    3) Preparation

    Cytopathology diagnosis is based on the observation of medical images at cellular level. The vast majority of tissue samples in the world are processed using formalin-fixed and Parrffin-embedded methods or frozen embedding via optimal cutting temperature compound,and then they are cut to multiple slices with a slicer at the thickness of around 10 μm. Notice that this process may spend 20~30 mins and is the bottleneck of intraoperative pathology.

    4) Imaging

    This step is the key step and will affect the following analysis. Existing WSI systems,e.g.,Hungary′s 3DHISTECH and German Leica,are widely used and still have to overcome several defects:low-quality imaging,low efficiency,narrow DOF,and high cost. These defects have been mentioned emphatically in Section 0 and Section 1. Despite the fact that the WSI scanners have existed for more than 20 years so far,the core requirements and bottlenecks of WSI instruments do not get the perfect solution before the birth of FPM.

    5) Analysis

    Conventional medical image analysis is dominated by manual operation,so there has to be a big input of professional pathologists. Additionally,lots of repetitive work can lead to low-efficiency medical diagnosis.

    6) Output reports

    Due to the above prominent problems,it takes at least 3 days to obtain the final pathological report for routine examinations. For rare and perplexing diseases,the time required can reach up to 7~10 days.

    The FPM solution for the next generation digital pathologic imager cunningly solves the problems in the imaging section,whose various technical indexes have been listed in Fig. 8(a). It is achievable to obtain a spatial resolution of around 200 nm~1 000 nm,a FOV of around 10 mm(2× objective)or 5 mm(4× objective)diameter full-color FPM reconstructed image within 4 s at the DOF of around 0.3~0.5 mm. However,the invention of this instrument is not the end of research. More modifications and techniques can be incorporated into the front and rear processing of this system in order to address the industry problems of digital pathology. These combinations will not be a simple adding,since the FPM can provide multimodal information that traditional scanners cannot provide,such as quantitative phase images,polarization images,fluorescence images,virtual staining,non-destructive 3D information,etc.,which will greatly enrich the data dimension of subsequent AI analysis,thus making it possible to make important breakthroughs compared to the traditional 70% success rate or recognition rate.

    3.2 Intraoperative pathology

    Intraoperative pathology is usually based on the observation of a small piece of sample. The main constraint of current intraoperative pathology lies in the low efficiency during sample preparation. Besides,the quality of sample preparation completely depends on artificial experience,which requires that the instrument must have a high tolerance:if the thickness of slice exceeds the limit of instrument,the slice cannot be scanned successfully.

    Conventional frozen process cannot be used in intraoperative pathology due to its destructive effect and high time cost. FPM solution can provide high-quality quantitative phase images and effectively shorten the time of sample preparation. Related technologies include virtual staining(Fig. 8(c)),morphological judgment based on grayscale phase image(Fig. 8(d)),multimodal fusion imaging(Fig. 8(e)),in situ observation(Fig. 8(f)),etc. Specially,virtual staining is a rapidly developing method based on machine learning and various imaging modes such as autofluorescence imaging161,quantitative phase imaging162,etc. The distribution of mapping relationship between input and output is studied through training models,and finally the color patterns are obtained. Virtual staining can greatly save cost and manpower,and avoid the difference of staining results caused by the randomness of personnel and technology. With its standardized staining ability,multiple staining information from the same FOV can be integrated to highlight features related to diagnosis. Simultaneously,virtual staining allows to further stretch the region or microstructure of interest by mixed staining 163. In addition to generating targeted colors for unlabeled tissue,staining transformation164 can be performed,which enables pathologists to evaluate different tissue components without additional staining procedures. However,numerous trials are still required to prove the feasibility and effectiveness of virtual staining before applying it to real clinical trials. The identification of staining results should go through critical evaluation by board-certified pathologists. To ensure widespread adoption,the technology must be scaled up to the point where it can be used effectively in large healthcare systems and easily by technicians with limited training165.

    3.3 Auxiliary diagnosis

    At present,the judgment of pathological sections depends on the knowledge and experience of pathologists. For some hospitals extremely lack of experienced experts,auxiliary diagnosis can produce quick,universal information processing solutions and reliable second opinions,which is likely to greatly improve the routine pathologic experience. For example,a large amount of standardized data sources and information with high resolution of sections(over 1 gigapixel)provided by FPM system has the potential to promote the rapid development of AI-based auxiliary diagnosis.

    Auxiliary diagnosis conducts primary screening of slides by establishing accurate detection model without prior processing or manual labeling,which accelerates the process of sample analysis166. DL-based auxiliary diagnosis can further provide content recognition and promote medical image analysis. To solve the limited applications of DL-based pathological diagnosis due to the lack of interpretation,researchers attempt to design systems combined with multimode neural networks. These networks restrain each other to generate regions of interest and text descriptions that meet the clinicopathological reporting standards161-171. The final output is provided as direct evidence for review and visual examination to help reduce the variability of clinical decision making. However,the reliability of auxiliary diagnosis is highly determined by the quality of data. Better data quality does not depend solely on the judgment of human eyes,but much more depends on the computer vision. For some rare diseases,only 50% to 60% of success rate can be achieved by pathologists,while computers might make better judgments through massive data processing. In this sense,FPM solution can provide high-quality results,but we do not know how much can be improved by using FPM data,which can be studied in the future.

    3.4 Three-dimensional reconstruction

    Generally,for cancer detection and diagnosis,only 1~2 slides are observed for a tumor tissue after preliminary screening in histopathologic analysis. Limited pathological slides can make it difficult for pathologists to assess the tissue as a whole. 3D reconstruction enables improved insight into the features and details of inner tissue structures. Doctors can rotate the 3D reconstruction to fully understand the growth pattern of a tumor. A straightforward and classical method to realize 3D reconstruction is to obtain multiple consecutive 2D slides through digital microscope,and then do a z-stack to reconstruct a 3D volume172. This type of 3D reconstruction belongs to a destructive method,and it relies on the alignment,registration,and combination of multiple 2D slides. Recently,a nondestructive 3D reconstruction method termed Fourier Ptychographic Diffraction Tomography(FPDT)62 was reported without cutting slides of the sample and z-scanning,which combines the concept of FPM and Optical Diffraction Tomography(ODT). Since FPM captures the images of angular-varied illuminations,it already contains the 3D information within the 2D dataset. By synthesizing a series of LR intensity images captured in 3D Fourier space,FPDT realizes large-volume and wide-field 3D reconstruction of biological samples. The theory can be referred in Refs.[862]and will not be in detail here. Note that affected by the coherence length and optical scattering,the optical penetration depth of this kind of ODT method is around 100 μm. And the slicer can cut a slide up to 100 μm. Therefore,it is quite match between a sample from slicer and ODT method and for a thicker tumor tissue,the conventional destructive method can also be combined.

    3.5 Telepathology

    Telepathology can be of great importance to grass-roots hospitals due to lack of pathology department. It allows consultation from specialized pathologists and more importantly,facilitates medical diagnosis or even multidisciplinary meetings from anywhere. Teleconsultations provide the possibility for low-level hospitals to share good medical resources and consequently improve accessibility. However,a medical report film(digital image)tends to have a large size(around 10 Gbits per slide),thus,how to compress,store,transmit and display it with High Definition(HD)remains as an essential issue to be addressed. On the one hand,more powerful networks(e.g. 5G/6G)are required for high-speed and stable transmission. Even if a cloud server is adopted,the upload and download of data are still necessary. On the other hand,developing appropriative compression algorithms and customized formats(e.g. pyramid structure173)might greatly increase the efficiency of data transmission.

    3.6 Drug screening

    Commercial multi-well(96 or 384 wells)plate readers can perform impedance or absorbance measurements on the contents of the wells with 10s per plate and have significant applications in cell culture and monitoring and drug screening research. But they can only give a rough description of samples based on their nature and have very limited information. A new generation imaging-based multi-well plate imager can provide a wealth of information within cell cultures,such as individual cell’s morphology,integrity,vitality and its connections to neighboring cells,measurements of absorption,thickness and dispersion,visualization of chemical compositions and structure,trace of gene expression through specific biomarker methods. Currently,commercial 96-well plate cell culture imagers,e.g.,the iCyte of Thorlabs,can scan a plate with every 8 mins at 1.2 μm resolution,which is around 50 times longer than a non-imaging well plate reader. A high efficiency,low-cost multi-well plate microscopic system with good performance is very challenging.

    There are three main steps in the whole process:sample preparation,imaging,and analysis. Hundreds of 96-well samples can be prepared within 6~24 hours and the analysis software can also do its job in real time. The main bottleneck is the throughput of the imaging process,which is similar to the conventional scanner of digital pathology. Given the fact that the FPM is a high-throughput or high content technique,and it can achieve high-throughput imaging for a single well,we sequentially reported a 6-well and a 96-well in cell culture imaging and drug screening system based on parallel FPM108-109,termed 6 Eyes and 96 Eyes,respectively. The 96 Eyes is made up of a commercial LED array,96 low-cost plastic-molded 4×/0.23 NA objectives,96 CMOS sensors(1 944×2 592 pixels,1.75 μm pixel pitch)and 4 frame grabber boards interfaced with 4 GPU modules for parallel computing. The 96 objectives and camera chips are housed on a customized 96-in-1 sensor board. By lighting up the multiplexed LED array,the 96 Eyes is capable of simultaneous imaging of all wells on a 96-well plate rapidly within cell cultures a rate of 0.7 frame per second and displaying high resolution(1.2 μm)bright field and aberration-free phase images of 1.1 mm by 0.85 mm per condition at the extended Depth-Of-Field(DOF)of ± 50 μmwithin 90 seconds,and dual-channel fluorescence images within 30 seconds. Comparing the 96 Eyes system with existing commercial products with respect to the throughput,as shown in Fig. 9,it has the largest throughput in the world so far to the best of our knowledge. The performances and principle can be referred to Refs.[108-109].

    Comparisons of the throughput of existing commercial products

    Figure 9.Comparisons of the throughput of existing commercial products

    In addition,large biopharmaceutical companies and top Clinical Research Organizations(CROs)are increasingly using digital pathology to streamline the process of drug discovery,preclinical and clinical trials,since the digital pathology is accompanied by the quantitative analysis that are difficult to detect with the human eye. For example,only one marker of clinical significance in multiple markers or in diffuse staining features of multiple cell compartments. The complexity of such detection methods is increasing and promotes the development of digital pathology solutions that combine AI for classification and pattern recognition with advanced high-throughput image acquisition techniques and multimodal imaging(bright field,phase,fluorescence,or multi-spectrum)to identify morphologically relevant tissue types and individual cell regions,enabling quantification of staining intensity. This has led to the development of digital pathology systems that can provide clinically relevant diagnostic or prognostic scores by comparing sample analysis outputs with standard curves from clinical data167.

    3.7 Teaching

    Education and research will be the first several industries to adopt digital pathology. It has been applied in the whole field of histopathology education,from high school,undergraduate and graduate education to continuous professional development and external quality assurance. Each learner will observe exactly the same thing instead of similar slides from the same tissue block. Therefore,standardization of curriculum materials guarantees to provide each learner with the same learning opportunities and teaching quality. In addition,digital pathology helps to include rare cases in curriculum materials. A digital copy of slide is enough for a standard class teaching,while multiple slides are required in conventional teaching. Nowadays,digital pathology is playing a profound role in promoting education reform.

    3.8 Standards

    Clear and uniform standards are the key to ensuring service quality and reliability,and the same is true for digital pathology. At present,the standardization of digital pathology has not achieved the desired level. There is no international standard,national standard,and professional standard. There may be some enterprise standards,but they are not enough.

    First,at the preparation stage,whether to stain diseased tissues efficiently is directly related to subsequent observation and final diagnosis. Standardized labeling methods with stable quality help to improve the accuracy of medical diagnosis. Despite the existence of guidelines,the use of staining doses is highly dependent on human manipulation.

    Second,the accuracy of slice thickness depends on the quality of slicer. Note that the conventional scanner has shallow DOF. If the sample is slightly thicker,the scanner will be failed.

    Third,the utilization of AI in auxiliary diagnosis has a strong demand for standardized slide images. The instrument that ensures stable standard of slide-making and imaging is essential to the development of pathological AI algorithms. However,the images between different conventional scanners are completely different because of the different wavefront aberration,different chromatic aberration,different detectors,different noise level,and other conditions. Differences in training sets will result in completely different judgment of test results.

    Fourth,automatic diagnosis of AI greatly cuts down medical work of pathologists,but there are still technical difficulties in data processing and result interpretation. Some rare diseases are desperately short of samples,even if a pathologist cannot make an accurate judgment. Once the conditions are slightly deviated,the diagnosis results of AI can be quite different. Establishing standardized evaluation system of imaging quality can to some extent avoids the errors. However,AI diagnosis still needs to be reviewed by pathologists at present and the results should be given comprehensively in combination with other clinical data.

    4 Conclusion and outlook

    At the beginning,FPM is a high-throughput technique that breaks the trade-off between FOV and spatial resolution. Given its flexible setup without mechanical scanning and interferometric measurement,FPM has improved rapidly,which not only acts as a tool to obtain both HR and large FOV but is also regarded as a paradigm to solve a series of trade-off problems,e.g.,the trade-off between angular resolution and spatial resolution in light field imaging. And it may inspire to solve the trade-off between spectral resolution and spatial resolution in imaging spectrometer in the future. It has been written into the book of the Introduction to Fourier Optics(4th edition)by Goodman. The pursuit of researchers is usually twofold,one is to put forward new theories to summarize objective laws and wish them can be written into textbooks if they are extremely significant. The second is to develop products based on their theory to reform the world. The FPM technology has already been recognized by the academic community. Therefore,a large number of scholars began to explore its different applications and even report some product information someday.

    In this paper,the development trend of FPM technique in 9 aspects is comprehensively summarized in Fig. 2,including high-precision imaging,high-throughput imaging,high-speed or single shot imaging,3D or tomography imaging,mixed state decoupling,spectral dimension(color imaging to hyperspectral imaging),high dynamic range,system extension,and typical applications. Digital pathology is one of the earliest and the most successful applications of FPM. Several stages and recent progress for the next generation of digital pathological imager and analyzer were reviewed. We point out that it has entered the industrialization stage of “10-100” in this application direction. Several typical results are provided in Fig. 6,and the technical indexes are listed in Fig. 8. It is achievable to obtain a spatial resolution of around 200 nm~1 000 nm,a FOV of around 10 mm(2× objective)or 5 mm(4× objective)diameter full-color FPM reconstructed image within 4 s at the DOF of around 0.3~0.5 mm. We expect it to develop a prototype and be capable of mass scanning and digital analysis function within the next 1~2 years. Building a prototype or product based on FPM will not be the end of this direction for FPM,on the contrary,it only solves one industry issue of the whole process of digital pathology,and it still has many tough tasks to do as shown in Fig. 8. We discussed its potential for large-scale socio-economic benefits,and its high-quality and high-content full-color images and quantitative phase images may promote the intraoperative pathology and quantitative AI diagnosis,by combining the advanced virtual staining,multimodal fusion imaging,label-free observation in situ,non-destructive ODT,preliminary screening and recognition with AI,etc.

    This combination will not be a simple adding. The connotation of computational imaging is to predict or simulate the final results and to give a feedback optimization for the optical systems and decoding algorithm. Therefore,there will be a professional analysis algorithm for a specific disease rather than the universal algorithm,like a difference between the general computer and quantum computer. The multi-dimensional information based on FPM will also enhance the effect of AI analysis. As a typical interdisciplinary field,there are still unsatisfactory aspects,including scientific problems,technical problems,engineering problems and industrial problems,which need joint efforts of various parties. We believe that the industry problems will eventually be overcome or alleviated,and the instruments and algorithms will benefit mankind.

    References

    [1] N FARAHANI, A PARWANI, L PANTANOWITZ. Whole slide imaging in pathology: advantages, limitations, and emerging perspectives. Pathology and Laboratory Medicine International, 7, 23-33(2015).

    [2] W S JAHN, M PLASS, F MOINFAR. Digital pathology: Advantages, limitations and emerging perspectives. Journal of Clinical Medicine, 9, 3697(2020).

    [3] G ZHENG, R HORSTMEYER, C YANG. Wide-field, high-resolution Fourier ptychographic microscopy. Nature Photonics, 7, 739-745(2013).

    [4] G ZHENG. Breakthroughs in photonics 2013: Fourier ptychographic imaging. IEEE Photonics Journal, 6, 1-7(2014).

    [5] Jiasong SUN, Yuzhen ZHANG, Qian CHEN et al. Fourier ptychographic microscopy: Theory, advances, and applications. Acta Optica Sinica, 36, 1011005(2016).

    [6] An PAN, Baoli YAO. High-throughput and fast-speed Fourier ptychographic microscopy. Infrared and Laser Engineering, 48, 603012(2019).

    [7] P C KONDA, L LOETGERING, K C ZHOU et al. Fourier ptychography: current applications and future promises. Optics Express, 28, 9603-9630(2020).

    [8] A PAN, C ZUO, B YAO. High-resolution and large field-of-view Fourier ptychographic microscopy and its applications in biomedicine. Reports on Progress in Physics, 83, 096101(2020).

    [9] G ZHENG, C SHEN, S JIANG et al. Concept, implementations and applications of Fourier ptychography. Nature Reviews Physics, 3, 207-223(2021).

    [10] Shaohui ZHANG, Guocheng ZHOU, Baiqi CUI et al. Review of Fourier ptychographic microscopy: Models, algorithms, and systems. Laser & Optoelectronics Progress, 58, 1400001(2021).

    [11] H M L FAULKNER, J M RODENBURG. Movable aperture lensless transmission microscopy: A novel phase retrieval algorithm. Physical Review Letters, 93, 023903(2004).

    [12] An PAN, Yan ZHANG, Tianyu ZHAO et al. Quantitative phase microscopy based on ptychography. Laser & Optoelectronics Progress, 54, 040001(2017).

    [13] An PAN, Xiaofei ZHANG, Bin WANG et al. Experimental study on three-dimensional ptychography for thick sample. Acta Physica Sinica, 65, 014204(2016).

    [14] An PAN, Dong WANG, Yishi SHI et al. Incoherent ptychography in Fresnel domain with simultaneous multi-wavelength illumination. Acta Physica Sinica, 65, 124201(2016).

    [15] V MICO, Z ZALEVSKY, P GARCÍA-MARTÍNEZ et al. Synthetic aperture superresolution with multiple off-axis holograms. Journal of the Optical Society of America A, 23, 3162-3170(2006).

    [16] J HOLLOWAY, Y WU, M K SHARMA et al. SAVI: Synthetic apertures for long-range, subdiffraction-limited visible imaging using Fourier ptychography. Science Advances, 3, e1602564(2017).

    [17] R W GERCHBERG, W O SAXTON. A practical algorithm for the determination of the phase from image and diffraction plane pictures. Optik, 35, 237-249(1972).

    [18] J R FIENUP. Phase retrieval algorithms: a comparison. Applied Optics, 21, 2758-2769(1982).

    [19] J W GOODMAN. Introduction to Fourier Optics (4th Ed)(2017).

    [20] J WU, Z LU, D JIANG et al. Iterative tomography with digital adaptive optics permits hour-long intravital observation of 3D subcellular dynamics at millisecond scale. Cell, 184, 1-15(2021).

    [21] Z BIAN, S DONG, G ZHENG. Adaptive system correction for robust Fourier ptychographic imaging. Optics Express, 21, 32400-32410(2013).

    [22] X OU, G ZHENG, C YANG. Embedded pupil function recovery for Fourier ptychographic microscopy. Optics Express, 22, 4960-4972(2014).

    [23] S DONG, Z BIAN, R SHIRADKAR et al. Sparsely sampled Fourier ptychography. Optics Express, 22, 5455-5464(2014).

    [24] K GUO, S DONG, P NANDA et al. Optimization of sampling pattern and the design of Fourier ptychographic illuminator. Optics Express, 23, 6171-6180(2015).

    [25] L-H YEH, J DONG, J ZHONG et al. Experimental robustness of Fourier ptychography phase retrieval algorithms. Optics Express, 23, 33214-33240(2015).

    [26] L BIAN, J SUO, G ZHENG et al. Fourier ptychographic reconstruction using Wirtinger flow optimization. Optics Express, 23, 4856-4866(2015).

    [27] Y ZHANG, W JIANG, Q DAI. Nonlinear optimization approach for Fourier ptychographic microscopy. Optics Express, 23, 33822-33835(2015).

    [28] L BIAN, J SUO, J CHUNG et al. Fourier ptychographic reconstruction using Poisson maximum likelihood and truncated Wirtinger gradient. Scientific Reports, 6, 27384(2016).

    [29] L BIAN, G ZHENG, K GUO et al. Motion-corrected Fourier ptychography. Biomedical Optics Express, 7, 4543-4553(2016).

    [30] J SUN, Q CHEN, Y ZHANG et al. Efficient positional misalignment correction method for Fourier ptychographic microscopy. Biomedical Optics Express, 7, 1336-1350(2016).

    [31] C ZUO, J SUN, Q CHEN. Adaptive step-size strategy for noise-robust Fourier ptychographic microscopy. Optics Express, 24, 20724-20744(2016).

    [32] Y ZHANG, A PAN, M LEI et al. Data preprocessing methods for robust Fourier ptychographic microscopy. Optical Engineering, 56, 123107(2017).

    [33] A PAN, Y ZHANG, T ZHAO et al. System calibration method for Fourier ptychographic microscopy. Journal of Biomedical Optics, 22, 096005(2017).

    [34] Y FAN, J SUN, Q CHEN et al. Adaptive denoising method for Fourier ptychographic microscopy. Optics Communications, 404, 23-31(2017).

    [35] A MAIDEN, D JOHNSON, P LI. Further improvements to the ptychographical iterative engine. Optica, 4, 736-745(2017).

    [36] A PAN, C ZUO, Y XIE et al. Vignetting effect in Fourier ptychographic microscopy. Optics and Lasers in Engineering, 120, 40-48(2019).

    [37] P SONG, S JIANG, H ZHANG et al. Full-field Fourier ptychography (FFP): spatially varying pupil modeling and its application for rapid field-dependent aberration metrology. APL Photonics, 4, 050802(2019).

    [38] A WANG, Z ZHANG, S WANG et al. Fourier Ptychographic Microscopy via Alternating Direction Method of Multipliers. Cells, 11, 1512(2022).

    [39] X OU, R HORSTMEYER, G ZHENG et al. High numerical aperture Fourier ptychography: principle, implementation and characterization. Optics Express, 23, 3472-3491(2015).

    [40] Z F PHILLIPS, M V D’AMBROSIO, T LEI et al. Multi-contrast imaging and digital refocusing on a mobile microscope with a domed LED array. PLoS One, 10, e0124938(2015).

    [41] Z F PHILLIPS, R ECKERT, L WALLER. Quasi-dome: A self-calibrated high-NA led illuminator for Fourier ptychography imaging systems and applications, IW4E.5(2017).

    [42] J SUN, C ZUO, L ZHANG et al. Resolution-enhanced Fourier ptychographic microscopy based on high-numerical-aperture illuminations. Scientific Reports, 7, 1187(2017).

    [43] A PAN, Y ZHANG, K WEN et al. Subwavelength resolution Fourier ptychography with hemispherical digital condensers. Optics Express, 26, 23119-23131(2018).

    [44] L BIAN, J SUO, G SITU et al. Content adaptive illumination for Fourier ptychography. Optics Letters, 39, 6648-6651(2014).

    [45] Y ZHANG, W JIANG, L TIAN et al. Self-learning based Fourier ptychographic microscopy. Optics Express, 23, 18471-18486(2015).

    [46] L TIAN, X LI, K RAMCHANDRAN et al. Multiplexed coded illumination for Fourier ptychography with an LED array microscope. Biomedical Optics Express, 5, 2376-2389(2014).

    [47] L TIAN, Z LIU, L H YEH et al. Computational illumination for high-speed in vitro Fourier ptychographic microscopy. Optica, 2, 904-911(2015).

    [48] X LIN, J WU, Q DAI et al. Camera array based light field microscopy. Biomedical Optics Express, 6, 3179-3189(2015).

    [49] T NGUYEN, Y XUE, Y LI et al. Deep learning approach for Fourier ptychography microscopy. Optics Express, 26, 26470-26484(2018).

    [50] Y XUE, S CHENG, Y LI et al. Reliable deep-learning-based phase imaging with uncertainty quantification. Optica, 6, 618-629(2019).

    [51] B LEE, J HONG, D YOO et al. Single-shot phase retrieval via Fourier ptychographic microscopy. Optica, 5, 976-983(2018).

    [52] X HE, C LIU, J ZHU. Single-shot Fourier ptychography based on diffractive beam splitting. Optics Letters, 43, 214-217(2018).

    [53] W LEE, D JUNG, S RYU et al. Single-exposure quantitative phase imaging in color-coded LED microscopy. Optics Express, 25, 8398-8411(2017).

    [54] Z F PHILLIPS, M CHEN, L WALLER. Single-shot quantitative phase microscopy with color-multiplexed differential phase contrast (cDPC). PloS One, 12, e0171228(2017).

    [55] J SUN, Q CHEN, J ZHANG et al. Single-shot quantitative phase microscopy based on color-multiplexed Fourier ptychography. Optics Letters, 43, 3365-3368(2018).

    [56] J SUN, C ZUO, J ZHANG et al. High-speed Fourier ptychographic microscopy based on programmable annular illuminations. Scientific Reports, 8, 7669(2018).

    [57] A PAN, C SHEN, B YAO et al. Single-shot Fourier ptychographic microscopy via annular monochrome LED array. Frontiers in Optics + Laser Science APS/DLS, FTh3F.4(2019).

    [58] P C KONDA, J M TAYLOR, A R HARVEY. Multi-aperture Fourier ptychographic microscopy, theory and validation. Optics and Lasers in Engineering, 138, 106410(2021).

    [59] L TIAN, L WALLER. 3D intensity and phase imaging from light field measurements in an LED array microscope. Optica, 2, 104-111(2015).

    [60] R HORSTMEYER, J CHUNG, X OU et al. Diffraction tomography with Fourier ptychography. Optica, 3, 827-835(2016).

    [61] A PAN, B YAO. Three-dimensional space optimization for near-field ptychography. Optics Express, 27, 5433-5446(2019).

    [62] C ZUO, J SUN, A ASUNDI et al. Wide-field high-resolution 3D microscopy with Fourier ptychographic diffraction tomography. Optics and Lasers in Engineering, 128, 106003(2020).

    [63] J LI, A MATLOCK, Y LI et al. Resolution-enhanced intensity diffraction tomography in high numerical aperture label-free microscopy. Photonics Research, 8, 1818-1826(2020).

    [64] N ZHOU, J LI, J SUN et al. Single-exposure 3D label-free microscopy based on color-multiplexed intensity diffraction tomography. Optics Letters, 47, 969-972(2022).

    [65] P THIBAULT, A MENZEL. Reconstructing state mixtures from diffraction measurements. Nature, 494, 68-71(2013).

    [66] J ZHONG, T LEI, J DAUWELS et al. et al 2014 Partially coherent phase imaging with simultaneous source recovery. Biomedical Optics Express, 6, 257-265(2014).

    [67] S DONG, R SHIRADKAR, P NANDA et al. Spectral multiplexing and coherent-state decomposition in Fourier ptychographic imaging. Biomedical Optics Express, 5, 1757-1767(2014).

    [68] R KARL, C BEVIS, R LOPEZ-RIOS et al. Spatial, spectral, and polarization multiplexed ptychography. Optics Express, 23, 30250-30258(2015).

    [69] J DENG, Y NASHED, S CHEN et al. Continuous motion scan ptychography: characterization for increased speed in coherent x-ray imaging. Optics Express, 23, 5438-5451(2015).

    [70] P LI, T EDO, D BATEY et al. Breaking ambiguities in mixed state ptychography. Optics Express, 24, 9038-9052(2016).

    [71] S DONG, K GUO, S JIANG et al. Recovering higher dimensional image data using multiplexed structured illumination. Optics Express, 23, 30393-30398(2015).

    [72] J SUN, Q CHEN, Y ZHANG et al. Sampling criteria for Fourier ptychographic microscopy in object space and frequency space. Optics Express, 24, 15765-15781(2016).

    [73] Y ZHOU, J WU, Z BIAN et al. Fourier ptychographic microscopy using wavelength multiplexing. Journal of Biomedical Optics, 22, 066006(2017).

    [74] M WANG, Y ZHANG, Q CHEN et al. A color-corrected strategy for information multiplexed Fourier ptychographic imaging. Optics Communications, 405, 406-411(2017).

    [75] K WAKONIG, A DIAZ, A BONNIN et al. X-ray Fourier ptychography. Science Advances, 5, eaav0282(2019).

    [76] Y GAO, J CHEN, A WANG et al. High-throughput fast full-color digital pathology based on Fourier ptychographic microscopy via color transfer. Science China-Physics Mechanics & Astronomy, 64, 114211(2021).

    [77] L LOETGERING, X LIU, A DE BEURS et al. Extreme ultraviolet multispectral ptychography with minimum entropy beams. Optica, 8, 130-138(2021).

    [78] X YANG, P C KONDA, S XU et al. Quantized Fourier ptychography with binary images from SPAD cameras. Photonics Research, 9, 1985-1969(2021).

    [79] C KUANG, Y MA, R ZHOU et al. Digital micromirror device-based laser-illumination Fourier ptychographic microscopy. Optics Express, 23, 26999-27010(2015).

    [80] J CHUNG, H LU, X OU et al. Wide-field Fourier ptychographic microscopy using laser illumination source. Biomedical Optics Express, 7, 4787-4802(2016).

    [81] S CHOWDHURY, M CHEN, R ECKERT et al. High-resolution 3D refractive index microscopy of multiple-scattering samples from intensity images. Optica, 6, 1211-1219(2019).

    [82] K GUO, Z BIAN, S DONG et al. Microscopy illumination engineering using a low-cost liquid crystal display. Biomedical Optics Express, 6, 574-579(2015).

    [83] S DONG, R HORSTMEYER, R SHIRADKAR et al. Aperture-scanning Fourier ptychography for 3D refocusing and super-resolution macroscopic imaging. Optics Express, 22, 13586-13599(2014).

    [84] X OU, J CHUNG, R HORSTMEYER et al. Aperture scanning Fourier ptychographic microscopy. Biomedical Optics Express, 7, 3140-3150(2016).

    [85] P SONG, S JIANG, H ZHANG et al. Super-resolution microscopy via ptychographic structured modulation of a diffuser. Optics Letters, 44, 3645-3648(2019).

    [86] H ZHANG, S JIANG, J LIAO et al. Near-field Fourier ptychography: super-resolution phase retrieval via speckle illumination. Optics Express, 27, 7498-7512(2019).

    [87] S DONG, P NANDA, K GUO et al. Incoherent Fourier ptychographic photography using structured light. Photonics Research, 3, 19-23(2015).

    [88] S DONG, P NANDA, R SHIRADKAR et al. High-resolution fluorescence imaging via pattern-illuminated Fourier ptychography. Optics Express, 22, 20856-20870(2014).

    [89] K GUO, Z ZHANG, S JIANG et al. 13-fold resolution gain through turbid layer via translated unknown speckle illumination. Biomedical Optics Express, 9, 260-274(2018).

    [90] M ZHOU, R LI, T PENG et al. Retrieval of non-sparse objects through scattering media beyond the memory effect. Journal of Optics, 22, 085606(2020).

    [91] S DONG, K GUO, P NANDA et al. FPscope: a field-portable high-resolution microscope using a cellphone lens. Biomedical Optics Express, 5, 3305-3310(2014).

    [92] S PACHECO, B SALAHIEH, T MILSTER et al. Transfer function analysis in epi-illumination Fourier ptychography. Optics Letters, 40, 5343-5346(2015).

    [93] K GUO, S DONG, G ZHENG. Fourier ptychography for brightfield, darkfield, reflective, multi-slice, and fluorescence imaging. IEEE Journal of Selected Topics in Quantum Electronics, 22, 77-88(2016).

    [94] S PACHECO, G ZHENG, R LIANG. Reflective Fourier ptychography. Journal of Biomedical Optics, 21, 026010(2016).

    [95] H LEE, B H CHON, H K AHN. Reflective Fourier ptychographic microscopy using a parabolic mirror. Optics Express, 27, 34382-34391(2019).

    [96] Z LI, J ZHANG, X WANG et al. High resolution integral holography using Fourier ptychographic approach. Optics Express, 22, 31935-31947(2014).

    [97] L H YEH, S CHOWDHURY, L WALLER. Computational structured illumination for high-content fluorescence and phase microscopy. Biomedical Optics Express, 10, 1978-1998(2019).

    [98] L H YEH, S CHOWDHURY, N A REPINA et al. Speckle-structured illumination for 3D phase and fluorescence computational microscopy. Biomedical Optics Express, 10, 3635-3653(2019).

    [99] Z LIU, L TIAN, S LIU et al. Real-time brightfield, darkfield, and phase contrast imaging in a light-emitting diode array microscope. Journal of Biomedical Optics, 19, 106002(2014).

    [100] T AIDUKAS, R ECKERT, R A HARVEY et al. Low-cost, sub-micron resolution, wide-field computational microscopy using opensource hardware. Scientific Reports, 9, 7457(2019).

    [101] J CHUNG, J KIM, X OU et al. Wide field-of-view fluorescence image deconvolution with aberration-estimation from Fourier ptychography. Biomedical Optics Express, 7, 352-368(2016).

    [102] X OU, R HORSTMEYER, C YANG et al. Quantitative phase imaging via Fourier ptychographic microscopy. Optics Letters, 38, 4845-4848(2013).

    [103] L TIAN, L WALLER. Quantitative differential phase contrast imaging in an LED array microscope. Optics Express, 23, 11394-11403(2015).

    [104] H LU, J CHUNG, X OU et al. Quantitative phase imaging and complex field reconstruction by pupil modulation differential phase contrast. Optics Express, 24, 25345-25361(2016).

    [105] C SHEN, M LIANG, A PAN et al. Non-iterative complex wave-field reconstruction based on Kramers-Kronig relations. Photonics Research, 9, 1003-1012(2021).

    [106] C GUO, Z BIAN, S JIANG et al. OpenWSI: a low-cost, high-throughput whole slide imaging system via single-frame autofocusing and open-source hardware. Optics Letters, 45, 260-263(2020).

    [107] J HUANG, A PAN, H JIN et al. Non-interferometric accurate phase imaging via linear-convergence iterative optimization. Optics and Lasers in Engineering, 144, 106630(2021).

    [108] J KIM, B M HENLEY, C H KIM et al. Incubator embedded cell culture imaging system (EmSight) based on Fourier ptychographic microscopy. Biomedical Optics Express, 7, 3097-3110(2016).

    [109] A CHAN, J KIM, A PAN et al. Parallel Fourier ptychographic microscopy for high-throughput screening with 96 cameras (96 Eyes). Scientific Reports, 9, 11114(2019).

    [110] A WILLIAMS, J CHUNG, X OU et al. Fourier ptychographic microscopy for filtration-based circulating tumor cell enumeration and analysis. Journal of Biomedical Optics, 19, 066007(2014).

    [111] J CHUNG, G W MARTINEZ, K C LENCIONI et al. Computational aberration compensation by coded-aperture-based correction of aberration obtained from optical Fourier coding and blur estimation. Optica, 6, 647-661(2019).

    [112] C SHEN, A CHAN, J CHUNG et al. Computational aberration correction of VIS-NIR multispectral imaging microscopy based on Fourier ptychography. Optics Express, 27, 24923-24937(2019).

    [113] J ZHANG, Z WANG, T LI et al. 3D object hiding using three-dimensional ptychography. Journal of Optics, 18, 095701(2016).

    [114] A PAN, K WEN, B YAO. Linear space-variant optical cryptosystem via Fourier ptychography. Optics Letters, 44, 2032-2035(2019).

    [115] J HOLLOWAY, M S ASIF, M J SHARMA et al. Toward long-distance subdiffraction imaging using coherent camera arrays. IEEE Transactions on Computational Imaging, 2, 251-265(2016).

    [116] M XIANG, A PAN, Y ZHAO et al. Coherent synthetic aperture imaging for visible remote sensing via reflective Fourier ptychography. Optics Letters, 46, 29-32(2021).

    [117] J M RODENBURG. Advances in Imaging and Electron Physics, 150, 87-184(2008).

    [118] J M RODENBURG, H M L FAULKNER. A phase retrieval algorithm for shifting illumination. Applied Physics Letters, 85, 4795-4797(2004).

    [119] P THIBAULT, M DIEROLF, O BUNK et al. Probe retrieval in ptychographic coherent diffractive imaging. Ultramicroscopy, 109, 338-343(2009).

    [120] A M MAIDEN, J M RODENBURG. An improved ptychographical phase retrieval algorithm for diffractive imaging. Ultramicroscopy, 109, 1256-1262(2009).

    [121] P THIBAULT, M GUIZAR-SICAIROS. Maximum-likelihood refinement for coherent diffractive imaging. New Journal of Physics, 14, 063004(2012).

    [122] S MARCHESINI, H KRISHNAN, B J DAURER et al. SHARP: a distributed GPU-based ptychographic solver. Journal of Applied Crystallography, 49, 1245-1252(2016).

    [123] Z WEN, C YANG, X LIU et al. Alternating direction methods for classical and ptychographic phase retrieval. Inverse Problems, 28, 115010(2012).

    [124] S MARCHESINI, A SCHIROTZEK, C YANG et al. Augmented projections for ptychographic imaging. Inverse Problems, 29, 115009(2013).

    [125] R HORSTMEYER, R Y CHEN, X OU et al. Solving ptychography with a convex relaxation. New Journal of Physics, 17, 053044(2015).

    [126] H CHANG, P ENFEDAQUE, S MARCHESINI. Blind ptychographic phase retrieval via convergent alternating direction method of multipliers. SIAM Journal on Imaging Sciences, 12, 153-185(2019).

    [127] A PAN, M ZHOU, Y ZHANG et al. Adaptive-window angular spectrum algorithm for near-field ptychography. Optics Communications, 430, 73-82(2019).

    [128] S ASLAN, V NIKITIN, D J CHING et al. Joint ptycho-tomography reconstruction through alternating direction method of multipliers. Optics Express, 27, 9128-9143(2019).

    [129] J M RODENBURG, A C HURST, A G CULLIS et al. Hard-X-ray lensless imaging of extended objects. Physical Review Letters, 98, 034801(2007).

    [130] P THIBAULT, M DIEROLF, A MENZEL et al. High-resolution scanning X-ray diffraction microscopy. Science, 321, 379-382(2008).

    [131] M DIEROLF, A MENZEL, P THIBAULT et al. Ptychographic X-ray computed tomography at the nanoscale. Nature, 467, 436-439(2010).

    [132] M GUIZAR-SICAIROS, A DIAZ, M HOLLER et al. Phase tomography from X-ray coherent diffractive imaging projections. Optics Express, 19, 21345-21357(2011).

    [133] M HOLLER, A DIAZ, M GUIZAR-SICAIROS et al. X-ray ptychographic computed tomography at 16 nm isotropic 3D resolution. Scientific Reports, 4, 3857(2014).

    [134] M V HOLT, S O HRUSZKEWYCZ, C E MURRAY et al. Strain imaging of nanoscale semiconductor heterostructures with X-ray Bragg projection ptychography. Physical Review Letters, 112, 165502(2014).

    [135] M STOCKMAR, I ZANETTE, M DIEROLF et al. X-ray near-field ptychography for optically thick specimens. Physical Review Applied, 3, 014005(2015).

    [136] M STOCKMAR, M HUBERT, M DIEROLF et al. X-ray nanotomography using near-field ptychography. Optics Express, 23, 12720-12731(2015).

    [137] J DENG, D J VINE, S CHEN et al. Simultaneous cryo X-ray ptychographic and fluorescence microscopy of green algae. Proceedings of the National Academy of Sciences of the United States of America, 112, 2314-2319(2015).

    [138] S HRUSZKEWYCZ, M ALLAIN, M HOLT et al. High-resolution three-dimensional structural microscopy by single-angle Bragg ptychography. Nature Mater, 16, 244-251(2017).

    [139] M HOLLER, M GUIZAR-SICAIROS, E TSAI et al. High-resolution non-destructive three-dimensional imaging of integrated circuits. Nature, 543, 402-406(2017).

    [140] P FRANZ. X-ray ptychography. Nature Photonics, 12, 9-17(2018).

    [141] A M MAIDEN, J M RODENBURG, M J HUMPHRY. Optical ptychography: a practical implementation with useful resolution. Optics Letters, 35, 2585-2587(2010).

    [142] Y SHI, T LI, Y WANG et al. Optical image encryption via ptychography. Optics Letters, 38, 1425-1427(2013).

    [143] M ODSTRCIL, J BUSSMANN, D RUDOLF et al. Ptychographic imaging with a compact gas–discharge plasma extreme ultraviolet light source. Optics Letters, 40, 5574-5577(2015).

    [144] P SIDORENKO, O COHEN. Single-shot ptychography. Optica, 3, 9-14(2016).

    [145] R HORSTMEYER, C YANG. A phase space model of Fourier ptychographic microscopy. Optics Express, 22, 338-358(2014).

    [146] M N LANDAUER. Indirect modes of coherent imaging in high-resolution transmission electron microscopy(1996).

    [147] C J SCHWARZ, Y KUZNETSOVA, S R J BRUECK. Imaging interferometric microscopy. Optics Letters, 28, 1424-1426(2003).

    [148] Y KUZNETSOVA, A NEUMANN, S R J BRUECK. Imaging interferometric microscopy. Journal of the Optical Society of America A, 25, 811-822(2008).

    [149] M P EDGAR, G M GIBSON, M J PADGETT. Principles and prospects for single-pixel imaging. Nature Photonics, 13, 13-20(2019).

    [150] G BARBASTATHIS, A OZCAN, G SITU. On the use of deep learning for computational imaging. Optica, 6, 921-943(2019).

    [151] L BARISONI, K J LAFATA, S M HEWITT et al. Digital pathology and computational image analysis in nephropathology. Nature Reviews Nephrology, 16, 669-685(2020).

    [152] D FACCIO, A VELTEN, G WETZSTEIN. Non-line-of-sight imaging. Nature Reviews Physics, 2, 318-327(2020).

    [153] D DAN, M LEI, B YAO et al. DMD-based LED-illumination Super-resolution and optical sectioning microscopy. Scientific Reports, 3, 1116(2013).

    [154] D DAN, B YAO, M LEI. Structured illumination microscopy for super-resolution and optical sectioning. Chinese Science Bulletin, 59, 1291-1307(2014).

    [155] J QIAN, M LEI, D DAN et al. Full-color structured illumination optical sectioning microscopy. Scientific Reports, 5, 14513(2015).

    [156] J LIAO, Y JIANG, Z BIAN et al. Rapid focus map surveying for whole slide imaging with continuous sample motion. Optics Letters, 42, 3379-3382(2017).

    [157] Z BIAN, C GUO, S JIANG et al. Autofocusing technologies for whole slide imaging and automated microscopy. Journal of Biophotonics, 13, e202000227(2020).

    [158] X DAI, S XU, X YANG et al. Quantitative Jones matrix imaging using vectorial Fourier ptychography. arXiv preprint, arXiv(2021).

    [159] View Research . Digital Pathology Market Size & Trends Report, 2030. https://www.grandviewresearch.com/industry-analysis/digital-pathology-systems-market

    [161] Y RIVENSON, H WANG, Z WEI et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nature Biomedical Engineering, 3, 466-477(2019).

    [162] Y RIVENSON, T LIU, Z WEI et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light: Science & Applications, 8, 23(2019).

    [163] Y ZHANG, K HAAN, Y RIVENSON et al. Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light: Science & Applications, 9, 78(2020).

    [164] K HAAN, Y ZHANG, J E ZUCKERMAN et al. Deep learning-based transformation of H&E stained tissues into special stains. Nature Communications, 12, 4884(2021).

    [165] Y RIVENSON, K HAAN, W D WALLACE et al. Emerging advances to transform histopathology using virtual staining. BME Frontiers, 2020, 9647163(2020).

    [166] G CAMPANELLA, M G HANNA, L GENESLAW et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine, 25, 1301-1309(2019).

    [167] Z ZHANG, P CHEN, M MCGOUGH et al. Pathologist-level interpretable whole-slide cancer diagnosis with deep learning. Nature Machine Intelligence, 1, 236-245(2019).

    [168] G WETZSTEIN, A OZCAN, S GIGAN et al. Inference in artificial intelligence with deep optics and photonics. Nature, 588, 39-47(2020).

    [169] T LIU, Z WEI, Y RIVENSON et al. Deep learning-based color holographic microscopy. Journal of Biophotonics, 12, e201900107(2019).

    [170] K HAAN, Y RIVENSON, Y WU et al. Deep-Learning-Based Image Reconstruction and Enhancement in Optical Microscopy. Proceedings of the IEEE, 108, 30-50(2020).

    [171] J LI, J GARFINKEL, X ZHANG et al. Biopsy-free in vivo virtual histology of skin using deep learning. Light: Science & Applications, 10, 233(2021).

    [172] I JANSEN, M LUCAS, C D SAVCI-HEIJINK et al. Histopathology: ditch the slides, because digital and 3D are on show. World Journal of Urology, 36, 549-555(2018).

    [173] N GUO, W XIONG, Q WU et al. An efficient tile-pyramids building method for fast visualization of massive geospatial raster datasets. Advances in Electrical and Computer Engineering, 16, 3-8(2016).

    An PAN, Yuting GAO, Aiye WANG, Huiqin GAO, Caiwen MA, Baoli YAO. High-throughput Full-color Fourier Ptychographic Microscopy for the Next Generation of Digital Pathologic Imager and Analyser(Invited)[J]. Acta Photonica Sinica, 2022, 51(7): 0751408
    Download Citation