
- Photonics Research
- Vol. 12, Issue 8, 1627 (2024)
Abstract
1. INTRODUCTION
Cell analysis is one of the most powerful techniques in both scientific research and clinical diagnosis and treatment and has been playing an extremely important role in the fields of disease diagnosis [1–4], prognosis tracking [5–8], cell screening [9–11], and drug discovery [12–14]. Imaging flow cytometry has emerged as an important tool in cell analysis, due to its high-throughput cell analysis capabilities of 10,000 events per second (eps), achieving the capture of information on cell morphology, structure, and component distribution. However, the performance of imaging flow cytometry is highly dependent on the availability and specificity of fluorescent labels, which inevitably causes chemical damage or staining-induced modification of cellular properties. In addition, the performance of existing imaging flow cytometers based on image sensors such as a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) is limited by the shutter speed as well as the imaging frame rate, resulting in a trade-off between the image quality and detection throughput. The optical time-stretch (OTS) imaging flow cytometry based on a single-pixel photodetector developed in our previous work overcomes the limitations mentioned above and achieves a high throughput of 1,000,000 eps with a spatial resolution of 780 nm in a label-free manner [15–17]. OTS imaging flow cytometry, with its high throughput and label-free features, has been widely applied in large-scale single-cell analysis [4,18]. Nevertheless, as it only provides bright-field images of cells, the available cell image information is limited, presenting challenges to thoroughly gain insight into the properties of cells.
Quantitative phase imaging (QPI) [19,20] is a valuable imaging method to retrieve high-content quantitative cell data by measuring and analyzing the phase information of light passing through the cells. It has been widely used in cell investigations to provide label-free measurements with nanoscale sensitivity [19,21–23]. Fortunately, OTS imaging flow cytometry has been combined with QPI to form OTS–QPI flow cytometry [24–28], which enables the simultaneous acquisition of intensity and phase images of cells to enrich information gathering. OTS–QPI flow cytometry decodes time-stretched spectral interferograms of the incident pulses to reconstruct the intensity and quantitative phase images of cells, thereby obtaining additional phase information on the refractive index and dry mass. However, the massive data brought about by high throughput poses challenges to data acquisition transmission and storage, hindering real-time application of OTS–QPI flow cytometry. All-optical data compression using compressed sensing (CS) is an ideal method to solve massive data from the source, and its combination with OTS imaging flow cytometry has been successfully implemented [29–33]. This method allows the original data to be sampled at a sampling rate much lower than that required by the Nyquist–Shannon sampling theorem, thereby reducing the data volume by more than one order of magnitude without sacrificing imaging speed and accuracy. However, to the best of our knowledge, there are currently no reports on acquiring richer cell image information from CS-equipped OTS imaging flow cytometers.
To address the problems mentioned above, in this work we propose and experimentally demonstrate Fourier-domain-compressed optical time-stretch quantitative phase imaging flow cytometry. Specifically, we directly acquire the high-frequency Fourier spectrum of an interference signal to reconstruct intensity and quantitative phase images of cells simultaneously from the compressed data. To evaluate the performance of our method, we first experimentally measured the static polystyrene microspheres and corn root cross section at compression ratios from 50% to 20%. In addition, we further demonstrated the measurement of breast cancer cells flowing in a microfluidic channel at a flowing speed of 1 m/s with different compression ratios. The acquired images and quantitative analysis values indicated that the imaging system maintains high image quality at a compression ratio of 30%. Furthermore, to show the powerful practical applications of Fourier-domain-compressed optical time-stretch quantitative phase imaging flow cytometry, we demonstrated intelligent image-based drug response analysis of breast cancer cells with biophysical phenotypic features and a convolutional neural network (CNN) trained with 24,000 images, achieving a high accuracy of 95% in distinguishing drug-treated and drug-untreated cells. Our method provides a solution for acquiring rich cell image information with CS-equipped OTS imaging flow cytometry, greatly expanding its applications in precision medicine, microbiology, and environment monitoring, where both high-throughput and high-content cell analysis is required.
Sign up for Photonics Research TOC. Get the latest issue of Photonics Research delivered right to you!Sign up now
2. PRINCIPLES AND METHODS
A. Principles
The principle and procedures of the method described in this paper are shown in Fig. 1. First, the short pulse is stretched in the time domain to map the spectrum into a temporal data stream, which means the interfered information can be recorded in the temporal data stream. Then, similar to conventional quantitative phase imaging, the time-stretched pulse is split into a reference pulse and a signal pulse to generate an interfered pulse described as
Figure 1.Principle of the Fourier-domain-compressed optical time-stretch quantitative phase imaging flow cytometry. The upper part shows the time-domain waveforms of each process, while the lower part displays the frequency-domain representation of the compressed signal processing. Time-stretch: the short pulse is stretched in the time domain to map the spectrum into a temporal data stream. Interference: the time-stretched reference pulse beats with the time-stretched signal pulse to form an interfered signal. Compressed sampling: the interfered pulse is modulated by the sinusoidal wave with different frequencies and then is compressed in the time domain to collect the high-frequency Fourier coefficients.
So, according to Fourier transform, we design a series of sinusoidal waves
B. Methods
1. Detection Workflow
To verify the effectiveness of the proposed method, we follow the workflow illustrated in Fig. 2(a), which is divided into three parts: (i) sample preparation, (ii) cell imaging using Fourier-domain-compressed optical time-stretch quantitative phase imaging flow cytometry, and (iii) image analysis based on CNN. The first part elucidates the process for culturing and drug-treating of MCF-7 breast cancer cells. In the second part, we provide a detailed configuration description of the Fourier-domain-compressed optical time-stretch quantitative phase imaging flow cytometry. In the final part, CNN is utilized to analyze images and identify changes caused by drug treatments.
Figure 2.(a) Workflow of cell detection with the Fourier-domain-compressed optical time-stretch quantitative phase imaging flow cytometry. (b) Detailed flowchart for sample preparation. (c) Experimental setup of the Fourier-domain-compressed optical time-stretch quantitative phase imaging flow cytometry. SMF: single-mode fiber; AWG: arbitrary waveform generator; MZM: Mach–Zehnder modulator; OL: objective lens; DCF: dispersive compensation fiber; PD: photodetector; and OSC: oscilloscope. (d) Architecture of the Inception-V3 neural network.
2. Sample Preparation
The MCF-7 breast cancer cells employed in this experiment were obtained from the China Center for Type Culture Collection (CCTCC) and we selected olaparib (642-941-5, Aladdin) as the antitumor drug. The culture medium utilized herein comprised 90% DMEM (Gibco, Thermo Fisher), 10% FBS (Gibco, Thermo Fisher), and 1% penicillin-streptomycin (sterilized, Aladdin) while maintaining the entire cultivation environment at a temperature of 37°C and a
3. Experimental Setup of Imaging Flow Cytometry
The experimental system designed and constructed according to the principle is shown in Fig. 2(c). First, the entire system is based on the OTS–QPI flow cytometry. The ultrashort 1550 nm femtosecond pulse laser (PriTel FFL-FRPMFA) with a bandwidth of 15 nm and a repetition rate of 101.7 MHz is time-stretched from 400 fs to about 7.65 ns through passing a roll of 30 km single-mode fiber (SMF), which corresponds to the “time-stretch” part in Fig. 1. After that, the time-stretched pulse is split into two paths by a 90:10 beam splitter (Thorlabs,BS045), with the 90% part being the signal path, and the remaining being the reference path. The optical pulse of the signal path is subsequently spatially dispersed by a diffraction grating (Thorlabs GR25-0616) to separate the frequency components of the pulse and then is focused on the test cells in the hydrodynamic-focusing microfluidic chip by an objective lens (Mitutoyo 56-982). Next, the pulse containing sample information is collected and recombined into a 50:50 SMF coupler. At the same time, the reference light passes through the delay line we set up and then is coupled into the SMF coupler to form the time-stretched spectral interferograms with the signal path. The operation above corresponds to the “interference” part in Fig. 1. Subsequently, the interferometric signal is modulated by the 40 Gb/s Mach–Zehnder modulator (MZM), in which the modulated sinusoidal code is generated by the arbitrary waveform generator (AWG) (Keysight M8195A). Then, the time-stretched and modulated interference pulse is compressed by a section of dispersive compensation fiber (DCF) in the time-domain, detected by the 10 GHz photodetector (PD), and digitized by the oscilloscope (Keysight DSA91304A). Finally, the collected data eventually is recovered with the method we mention in the principle. After the system is set up, the center frequency of the high-frequency part of the interferometric signal is about 5 GHz, which is approximately 49 times the pulse frequency
4. Structure of the CNN
Detailed information on the CNN architecture utilizing the Inception-V3 framework [34] is shown in Fig. 2(d). When solely employing intensity image classification, the input terminal exclusively accepts intensity images. However, when incorporating the intensity and phase images for classification, the input terminal adopts a multimodal input approach. This Inception-V3 includes 42 layers, mainly consisting of nine inception modules. Each inception module comprises multiple convolutional layers and max pooling layers, with the convolutional layers employing diverse sizes of kernels to capture the image features of various scales, while the max pooling layers are used for dimensionality reduction. At the same time, the reduction assists the Inception-V3 in reducing the dimensionality of feature maps without losing critical information, thereby decreasing the computational complexity and enhancing the network’s computational efficiency. The use of dropout during training effectively mitigates overfitting, enabling superior generalization capabilities beyond the confines of the training dataset, thereby enhancing the model’s robustness and generalizability. All the experiments are conducted in PyTorch under a Windows server with an Intel Xeon(R) Gold 6143 CPU at 2.80 GHz, 256 GB of RAM, and one NVIDIA 1080Ti GPU with 11 GB memory.
3. RESULTS
A. Imaging the Static Polystyrene Microsphere and Corn Root Cross Section
To demonstrate the capability of Fourier-domain-compressed optical time-stretch quantitative phase imaging flow cytometry, we first imaged polystyrene microspheres and the corn root cross section with different compression ratios, where the microspheres served as a standard sample for evaluating the performance of the imaging system, and the corn root cross section, with finer features, was used to further quantify the detection capabilities of the imaging system. Here, microspheres and corn root cross section were fixed on the slide and moved with the motorized translation stage (Thorlabs, Z812B) at a speed of 1 μm/s in a direction orthogonal to the 1D optical beam. In this work, we experimentally imaged 10 polystyrene microspheres and one corn root cross section with compression ratios from 50% to 20%, and Figs. 3(a) and 3(b) illustrate the intensity and phase images of one of them. Specifically, Figs. 3(a) and 3(b) show the origin image acquired without compression, which is used as the reference. The results indicate that the intensity and phase images begin to show distortion compared to the original as the compression ratio is reduced to 30%, and the distortion increases significantly when the compression ratio is reduced to 20%. Therefore, the maximum compression ratio of the imaging system is about 30%.
Figure 3.Static performance of the imaging system. (a) and (b) Intensity and phase images of the polystyrene microsphere and corn root cross section under different compression ratios, respectively. (c) and (f) Phase curves along the dashed lines in (a) and (b), respectively. (d) and (e) Average value and standard deviation of SSIM and PSNR for the polystyrene microsphere intensity and phase images under different compression ratios. (g) and (h) SSIM and PSNR for the corn root cross section intensity and phase images under different compression ratios. Exp: experiment. Scale bar: 10 μm.
To quantitatively demonstrate the accuracy of the calculated phase values, we plot the phase curves for different compression ratios, shown in Figs. 3(c) and 3(f). Phase curves are along the dashed line in Figs. 3(a) and 3(b), which is perpendicular to the direction of motion of the microspheres. The results show that the phase curves at a compression ratio of 30% exhibit a high similarity to the curve of the original image, but then the curve match decreases as the compression ratio decreases. Moreover, to quantitatively evaluate the reconstructed image quality, we calculate the structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) values between the reconstructed images and the original ones of 10 microspheres at a different compression ratio and count their average value and standard deviation, where the results are shown in Figs. 3(d) and 3(e), respectively. It can be observed that with the increase in compression ratio, the average values show a decreasing trend in both intensity and phase images, indicating a significant loss of image information during compression. It is worth noting that the values of SSIM and PSNR decrease gently when the compression ratio increases from 50% to 30%, while the values decrease sharply when the compression ratio continues to decrease. Similarly, the values of SSIM and PSNR presented in Figs. 3(g) and 3(h) for the reconstructed intensity and phase images of the corn root cross section also indicate that the values decrease sharply when the compression ratio is below 30%. Additionally, the phase image quantitatively displays variations of the textures in different parts of the corn root cross section, which aids in analyzing the biocomponent differences. The quantitative results indicate that a 30% compression ratio is an appropriate value to balance the image quality and data compression volume, which agrees well with the image results.
B. Imaging the Flowing Breast Cancer Cells
In addition to the static experiments performed above, we further experimentally demonstrated the performance of the imaging flow cytometry in measuring flowing cells with a flowing speed of up to 1 m/s. The intensity and phase images of the breast cancer cells acquired at different compression ratios are shown in Fig. 4(a). It can be seen that the intensity and phase images are well reconstructed with a quality comparable to the original intensity and phase images at compression ratios between 50% and 30%, whereas as the compression ratio continues to increase, the edges and interiors of the intensity and phase images appear distorted. Similarly, to quantitatively evaluate the quality of the reconstructed images, we calculate the natural image quality evaluator (NIQE) as well as the Shannon entropy (SEN) values of 100 intensity and phase images obtained at each compression ratio and count their average value and standard deviation. Due to the lack of an available reference image for reconstructed images obtained at each compression ratio, we employ unreferenced image quality evaluation indicators NIQE and SEN to evaluate the quality of individual images, with smaller values indicating higher image quality [35]. The results are shown in Figs. 4(b)–4(e), where Figs. 4(b) and 4(c), respectively, represent the NIQE and SEN values of the intensity images, while Figs. 4(d) and 4(e) correspond to the NIQE and SEN values of the phase images. It is clear that the values of NIQE and SEN jump at a compression ratio of 30%, which is consistent with the trend observed in the images. These results again indicate that the appropriate compression ratio for the imaging system constructed in this work is 30%.
Figure 4.Imaging performance of flowing cells. (a) Intensity and phase images of the breast cancer cells under different compression ratios. (b)–(e) Average value and standard deviation of NIQE and SEN for the intensity and phase images acquired under different compression ratios. Scale bar: 10 μm.
C. Single-Cell Image-Based Biophysical Phenotyping Analysis in Breast Cancer Cells Drug Responses Testing
We have demonstrated the feasibility of the compressed phase imaging method we proposed through the aforementioned dynamic experimentation, which forms the foundation for our utilization of this imaging flow cytometry in real biomedical applications. Next, we applied the imaging flow cytometry to the drug response detection of breast cancer cells, intending to assist clinical drug use and pharmaceutical research. First, we conducted high-throughput detection of MCF-7 cells treated with different concentrations of an olaparib solution [control (0 μM), 50 μM, and 100 μM] using both compressed and uncompressed methods, respectively. Based on the analysis of Fig. 4, a compression ratio of 30% has been selected here. In this work, the sampling rate is set at 40 GSa/s without compressed sampling, so each pulse has about 400 points. The compression ratio of 30% corresponds to 120 points collected for each pulse, and the corresponding frequency range of the sinusoidal wave is
Figure 5.(a) and (b) Uncompressed and compressed MCF-7 cell images under drug concentrations of control, 50 μM, and 100 μM, respectively. (c) and (d), (e) and (f) The biophysical phenotypic single indicator analysis of uncompressed cell images and compressed cell images. (g)–(i) The dual index combination analysis of uncompressed intensity images, uncompressed intensity and phase images, and compressed intensity and phase images, respectively.
To further assess the impact of the drug on cells, we performed biophysical phenotypic analysis to evaluate the roundness, image entropy, and image contrast features of the cell intensity images, as well as the mean of phase, dry mass, and dry mass density features of the cell phase images, as defined in detail in Table 1. Moreover, to evaluate the cellular properties on the same scale, we normalized the feature values of biophysical phenotypic in addition to the roundness feature. The analysis results for representative biophysical phenotypic features are presented in Fig. 5, with additional results in Figs. 7–10 of Appendix A. As depicted in Fig. 5(c), with the escalation in the drug concentration, there is a diminution in the mean roundness of the cells, presenting an increasingly irregular morphology. Figure 5(d) illustrates that the cellular mean of phase decreases concomitantly with increasing concentrations of the drug. This is because autophagy induced by olaparib leads to cell death, which results in intracellular mitochondria being sequestered and degraded [36]. Figures 5(e) and 5(f) are the results of a single-index analysis of the compressed cell image, corresponding to Figs. 5(b) and 5(c). We find out that the biophysical phenotypic parameters of compressed cellular imagery exhibit an identical pattern of variation in trend to those of the uncompressed indices, proving that the imaging system proposed in this paper can effectively preserve the characteristics of cells, and clearly show the changes in cells. We proceeded to amalgamate two kinds of biophysical characterizations to carry label-free classification of the cells treated with varying concentrations of pharmaceuticals, and some of the results are shown in Figs. 5(g)–5(i). Figure 5(g) depicts the cellular classification results based on the parameters of image entropy and roundness derived from intensity images. It can be observed that the overlap between different types of cells cannot be resolvable. In contrast, the classification results that amalgamate phase and intensity exhibit that cells of different types aggregate at different positions within the coordinates in Fig. 5(h), and can effectively distinguish different types of cells. It suggests that the integration of phase imaging can offer a more comprehensive cellular characterization, enhancing the capability for classification. Meanwhile, we compare the results of Fig. 5(i) with Fig. 5(h) and find that they have similar results, further supporting the conclusion that compression effectively preserves cellular features.
Definitions of Symbols and Equations of the Biophysical Features
Parameter | Symbol | Unit | Equation |
---|---|---|---|
Quantitative phase | rad | – | |
Normalized light intensity | – | – | |
Perimeter | μm | – | |
Area | – | ||
Central wavelength | nm | 1550 | |
Volume | Vol | ||
Roundness | – | ||
Image entropy | – | ||
Image contrast | – | ||
Mean of phase | rad | ||
Dry mass | DM | pg | |
Dry mass density | DMD | pg/fL |
D. Intelligent Evaluation of Drug-Treated and Drug-Untreated Breast Cancer Cells
While biophysical phenotyping can distinguish between various cell types to some extent, achieving rapid and precise differentiation remains challenging. To achieve precise classification of cells treated with different drug concentrations, a deep convolutional neural network (CNN) was trained in this work. Considering the requirement of image classification, we opted for the Inception-V3 model, known for its superior classification performance and easy scalability. We have collected a total of 24,000 images (12,000 compressed and 12,000 uncompressed), and the irrelevance of the image data is displayed in Fig. 11 in Appendix B. Among the 12,000 images, there are 4000 images for each drug concentration, and the ratio of intensity and phase image numbers is 1:1. The 24,000 compressed and uncompressed images are acquired over three experiments, each yielding images of nontreated MCF-7 cells and drug-treated MCF-7 cells. Images obtained in each experiment are randomly split in a ratio of 3:1:1 to form the training, validation, and test sets. Consequently, the final training, validation, and test sets include image data from all the three experiments. After digital signal processing, the size of the input images is
Here, we perform cell classification using four different databases, respectively, which are uncompressed intensity images, uncompressed phase images, uncompressed intensity and phase images, as well as compressed intensity and phase images. As shown in Figs. 6(a)–6(d), we draw the
Figure 6.Performance of MCF-7 cells classification. Each row, from left to right, represents the classification results of uncompressed intensity images, uncompressed phase images, uncompressed intensity and phase images, and compressed intensity and phase images. (a)–(d) The
4. CONCLUSION
In this work, we propose and experimentally demonstrate Fourier-domain-compressed optical time-stretch quantitative phase imaging flow cytometry, which is capable of simultaneously acquiring intensity and quantitative phase images of cells from the compressed data. Specifically, we modulate a series of high-frequency sinusoidal waves on the time-stretched spectral interferograms and thus reconstruct the intensity and phase images by collecting the Fourier series of the high-frequency components and performing an inverse Fourier transform. With the setup above, we first explain the principle of its realization theoretically and then evaluate its performance through experimental measurement of static polystyrene microspheres and corn root cross section as well as flowing breast cancer cells at different compression ratios. The results of image and quantitative analysis show that the intensity and phase images acquired at a compression ratio of 30% are of comparable quality to the original image. Based on the basic performance of the imaging system, we further demonstrate its practical application capability in the drug response analysis of breast cancer cells. The biophysical phenotypic analysis of breast cancer cell intensity and phase images clearly illustrates the trend of changes in cellular biological properties with different drug concentrations. With the powerful data mining and analysis ability of the CNN, it is trained, validated, and tested on 24,000 intensity and phase images and achieves distinguishing drug-treated and drug-untreated cells with an accuracy of over 95%. In summary, the basic performance of the imaging system, as well as its performance in biological applications, demonstrates its promise for fast and precise cell analysis in both scientific research and clinical settings.
When combined with quantitative phase imaging, the CS-equipped OTS imaging flow cytometer not only enriches the cell information obtained but also retains the advantages of compressive sensing. During data acquisition, it requires a lower hardware sampling rate, reducing the cost of the imaging system and facilitating its widespread application. Meanwhile, the amount of data transmitted and stored by the system has been greatly reduced, which helps achieve real-time transmission of massive cell image data. In this work, when compressed sampling is not performed, the sampling rate of the imaging system is 40 GSa/s. While compressed sampling is performed, the sampling rate is equals to the repetition rate of the laser (i.e., 101.7 MHz); thus, the data rate has dropped by about
APPENDIX A: SINGLE-CELL IMAGE-BASED BIOPHYSICAL PHENOTYPING ANALYSIS IN BREAST CANCER CELLS DRUG RESPONSES TESTING
Figure
The results of the combined analysis of the biophysical phenotypic indicators of uncompressed strength images are showcased in Fig.
Figure 7.Biophysical phenotypic single indicator analysis of uncompressed cell images and compressed cell images. (a) and (b) The image contrast analysis and image entropy analysis of uncompressed cell intensity images. (c) and (d) The analysis of dry mass and dry mass density of uncompressed phase images. (e) and (f) The image contrast analysis and image entropy analysis of compressed cell intensity images. (g) and (h) The analysis of dry mass and dry mass density of compressed phase images.
Figure 8.(a) The biophysical phenotypic combination analysis of image contrast index and roundness index extracted from uncompressed intensity images. (b) The combination analysis of image entropy index and image contrast index extracted from uncompressed intensity images.
Figure 9.(a) The biophysical phenotypic combination analysis of dry mass index and image contrast index extracted from uncompressed phase and intensity images, respectively. (b) The biophysical phenotypic combination analysis of the mean of phase index and roundness index extracted from uncompressed phase and intensity images, respectively.
Figure 10.(a) The biophysical phenotypic combination analysis of dry mass index and image contrast index extracted from compressed phase and intensity images, respectively. (b) The biophysical phenotypic combination analysis of the mean of phase index and roundness index extracted from compressed phase and intensity images, respectively.
APPENDIX B: CORRELATION ANALYSIS OF THE TRAINING DATA OF THE MODEL
Figure
Figure 11.Correlation analysis of the training data of the model: (a) uncompressed cellular data and (b) compressed cellular data.
References
[34] C. Szegedy, V. Vanhoucke, S. Ioffe. Rethinking the inception architecture for computer vision. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2818-2826(2016).
[35] R. Schielein, S. Schröpfer, M. Kiunke. Quantitative evaluation of CT images by means of Shannon entropy. 11th European Conference on Non-Destructive Testing, 2-4(2014).
[37] V. D. M. Laurens, G. Hinton. Visualizing data using t-SNE. J. Mach. Learn. Res., 9, 2579-2605(2008).

Set citation alerts for the article
Please enter your email address