As a crucial component in life research, cellular analysis helps to reveal changes in cell differentiation, metabolism, gene expression^[1,2], etc. Among the various technical approaches available, the cell identification and sorting technologies provide a manner to extract individual cell or specific cell groups. Cell identification is the essential precondition for cell sorting. The typical identification methods include immunomagnetic beads^[3,4], cell surface or fluorescence labeling^[5], and so on. However, these methods have certain limitations (for instance, the essential pretreatment labeling of samples, and the random changes in the properties of target cells caused by the labels)^[6,7]. These defects thus affect their subsequent promotion and application.

The vision-based neural networks can directly use bright-field imaging data for cell classification^[8–10]. It not only overcomes the limitations of traditional cell separation techniques, but also improves the automation level of the system. Machine vision, which has been widely used in facial recognition^[11], autonomous driving^[12], and image detection^[13–16], is an extension of human perception and allows computers to extract image feature information through convolutional neural networks (CNNs) to obtain and perceive relevant information. By training with a large data set, such an intelligent model can achieve classification predictions within milliseconds. It thus greatly improves work efficiency by reducing time consumption in the identification process and eventually achieves efficient and accurate identification and location of the targets. However, it is only suitable for the classification prediction of a single cell, and is not well-suited in coping with situations where multiple targets exist in the field of view simultaneously.

As for the cell sorting, it is typically carried out by the feat of gravity^[17], centrifugal force^[18], and sound waves^[19–21]. In these methods, high throughput operation is possible, but certain cytoactive effects are inevitable, and precise manipulation is still a challenge. The optical tweezers technique, due to its noncontact and nondestructive characteristics, is a potent alternative for this topic^[22–24]. In this work, we demonstrate an artificial intelligence (AI)-assisted cell identification and sorting system in a microfluidic chip. In the system, we employ the you only look once (YOLO) object detection method based on a one-step CNN as the “brain” of the system. The YOLO model classifies all cell targets in mixture sample and accurately locates the target-cell’s position in the microfluidic channel at a millisecond order. After adaptive training, the average inference speed of the model is about 30 ms. On this basis, target cells are precisely sorted into an individual chamber by optical tweezers, which provides an independent space for cell cultivation and further research. This work provides a powerful platform for cell identification and sorting and is expected to play a role in future development of cellular studies.

Figure 1 depicts schematic of the AI-assisted optical system for cell identification and sorting. A 532 nm laser (MSL-FN-532-S, 400 mW) is first expanded to fit the size of a spatial light modulator (SLM) (PLUTO-2.1, 60 Hz, HOLOEYE), and then a spherical wave is generated by loading a predesigned hologram onto the SLM. To fine-tune the focal field, the zeroth-order diffraction beam is filtered, and the remaining first-order light field is modulated by a 4f system before its incidence into the objective (40×, NA=0.69). An attenuation device composed of a half-wave plate and a polarizer is inserted in front of the objective lens, where the half-wave plate is mounted on a precise rotating table to accurately modulate intensity of the focus field. A microfluidic channel chip is employed to inject a sample solution, where the inlet and outlet of the chip are connected to the microfluidic injection air pump, to keep the entire chip system working in a closed state. By adjusting the injection pressure, the flow rate of the solution inside the chip can be controlled. Finally, the image is captured by CCD camera after filtering the obtrusive laser signal and then is fetched by the AI program for target detection.

The microfluidic chip is made of polydimethylsiloxane (PDMS) bonding on a glass substrate, which includes five main channels and twenty L-shaped subchambers. The L-shaped chambers are designed to prevent cells from being washed out when the microfluidics are in a high flow rate state in the main channel. The mixture solution is prepared by mixing yeast cells and polystyrene (PS) spheres with a diameter of 5 µm so that they may possess similar sizes. The mixture solution is centrifuged twice and then diluted with deionized water to avoid blocking the channels.

Figure 2 depicts the experimental procedure of this work. First, deionized water is used to fill the microfluidic channel to drive out the air inside the channel and chambers [Fig. 2(a)]. Second, the mixture of yeast cells and PS spheres is injected at a rate of 1 nL/s into the chip [Fig. 2(b)], and the mixture solution will flow uniformly in the main channel sometime later. Afterwards, image detection of the yeast cells and PS spheres is performed [Fig. 2(c)]. Third, the YOLO algorithm is used to detect the samples in real time. In this process, the captured image is precompressed to a width of 1280 pixels. The black boxes in Fig. 2(d) indicate a sliding window, which sequentially selects a portion of the image for convolution with the convolutional kernel to extract similar features. The features information is extracted to generate feature vectors by a CNN. Subsequently, the feature vectors are used to infer the category probability and coordinate information of each target through a fully connected layer [Fig. 2(d)]. Here, each feature vector represents the feature information of an anchor box. In order to predict the target more accurately, every target is covered by multi-overlapped anchor boxes, and the box with the highest score is selected at the end after non-maximum suppression (NMS)^[25], as depicted in Fig. 2(e). Because of the prediction inference information of the AI algorithm, it takes only 30 ms to complete the inference prediction for each image frame. Based on the identification results, the yeast cells of interest can be proactively manipulated into the appointed subchamber by optical tweezers [Fig. 2(f)], thus achieving the goal of sorting individual cells from the mixture.

Based on the above AI-assisted object detection algorithms, identification and localization of yeast cells in a mixture solution are achievable, effectively improving the accuracy of the distinguishing ability for samples with similar morphological sizes. It should be noted that the acquired images may lose detailed information inside the cell when it is slightly out of focus, resulting in a feature loss and defective resolution. To address this issue, a data set containing bright-field images under various magnifications and focal lengths is precreated to enhance its adaptability. Additionally, in order to reduce the probability of misidentifying the background as particulate samples, images with microfluidic channels in the background are requisite in the training set. After 200 iterative training sessions, the total inference prediction and NMS time for a single image frame is about 30 ms. The confusion matrix of the algorithm model, as shown in Fig. 3(a), indicates that almost all cells and PS spheres can be correctly classified. The first column in the confusion matrix represents all true instances of identification of the cells, where 95% of them are correctly predicted, while 5% are falsely predicted as the background. Here, FN represents the false negatives, indicating the proportion of missed detections; while FP is the false positives, reflecting the proportion of misclassifying background as the target. Therefore, the data in the third column indicate that, in the case of erroneously identifying the background as targets, 69% are misidentified as cells and 31% as PS spheres. Significantly, background misidentification is a rare event. There the third column only represents the ratio of the two FP types. Generally, the mean average precision (mAP), which is defined as the integral of the area under the precision-recall curve^[26], is employed to measure the accuracy of a model. In our model, it has reached 0.98, being credible enough for target identification.

In the AI algorithm model, the video captured by the CCD camera is used as the input source to calibrate the identification efficiency in experiments (Visualization 1). Figure 3(b1) is the full view of a discretionarily selected frame from the video. For a clearer presentation, Figs. 3(b2) and 3(b3) are enlarged views from another two frames within the labeled area by the red box in Fig. 3(b1). Figure 3(c) shows the identification results of the samples corresponding to Fig. 3(b). For a better presentation, the bounding boxes surround the targets in Fig. 3(c) have been expanded by six pixels outward from the actual predicted box size, and the yeast cells and PS spheres are marked by blue and magenta boxes, respectively. This clearly indicates that even aggregated cells can be individually identified and accurately located. In Fig. 3(c), the numbers alongside the labels denote the identification scores of the targets, being composed of position scores and category scores. The score signifies the completeness of the target within the bounding box, which is the product of the presence score and the accuracy score of targets within the bounding box. Therefore, a higher score indicates a better confidence and recognition by the system for a given target. Experimentally, all identification scores are around 0.8, meaning that this model can accurately identify and locate the target cells.

The microfluidic chip preparation and operation can significantly impact the final identification performance, where bubbles and impurities could lead to a false result. To avoid these issues, here special precautions have been taken, including properly scoping the recognition range, enriching the data set, improving the quality of microfluidic chips, etc. In addition, the images captured by the CCD camera should be free from ghosting artifacts to be correctly recognized and detected. Hence, the maximum flow rate in experiments is jointly limited by the acquisition speed of the CCD camera and the inference time of the model.

Based on the above AI-assisted identification results, the identified yeast cells can be then sorted from the mixture and manipulated into the subchamber by the optical tweezers. The force of optical tweezers usually lies in the piconewton (pN) range; therefore, microfluidics in the channel maintain at a relatively low flow rate (<20μm/s) in the experimental sorting process. The flow rate is controlled by adjusting the air pressure at the inlet and outlet, and the solution in the chip channel flows smoothly and slowly. Figure 4 shows the experimental results of sorting the identified yeast cells (Visualization 2). In Figs. 4(a) and 4(b), the trapped cell inevitably moves upward due to the scattering force. As a result, it appears slightly out of focus. Nevertheless, our model still possesses a favorable adaptability because it has been pretrained with morphological information of numerous cells at different focal planes. Therefore, the trapped yeast cell can still be identified in real time during the manipulation process.

Figure 4 illustrates the entire process from cell recognition to capture. The green arrows indicate the cells that have been successfully trapped, and the red dots in Figs. 4(c) and 4(e) represent the moving trajectory of the trapped cell. By cycling the manipulation procedure, multiple cells of interest can be successfully sorted into the designated subchambers. Figure 4(f) shows the multiple sorting results, which verifies the feasibility and stability of our system.

In experiments, the entire channel undergoes a flow status. Therefore, during the optical sorting process, new cells will come into the view field. This indicates that conducting dynamic sorting is feasible. For future developments, the optical manipulation can be further combined with the automation control program to project the transport tracks of trapped cells. Furthermore, by virtue of the separate chambers, the sorted cells can be used for intensive research, such as cell interactions and cell proliferation.

In this work, we demonstrated an AI-assisted cell identification and sorting system on a chip. It provides an achievable approach to identify and transport target cells from a mixture solution. The AI-assisted arithmetic model implements accurate identification and localization of the targets successfully. On this basis, as it can simultaneously locate the positions of all targets in the view field, including the “obstacles,” it then guides the sorting process for cells of interest to get around. In this system, the trapping light field is modulated by the SLM, and dynamic concurrent manipulations are possible by generating switchable multiple holographic fields. In addition, by virtue of an adaptive dynamics programming control, an automatic system for cell identification and sorting is also viable. Overall, this work provides a prototype of automatic cell-sorting technique and will create opportunities for subsequent cell proliferation and other analytical works.