• Laser & Optoelectronics Progress
  • Vol. 58, Issue 22, 2217001 (2021)
Zhongfa Liu1、2, Yizhe Yang1、2, Yu Fang1、2, Xiaojing Wu3、**, Siwei Zhu3, and Yong Yang1、2、*
Author Affiliations
  • 1Institute of Modern Optics, Nankai University, Tianjin 300350, China
  • 2Tianjin Key Laboratory of Micro-Scale Optical Information Science and Technology, Tianjin 300350, China
  • 3Tianjin Union Medical Center, Institute of Translational Medicine, Nankai University, Tianjin 300121, China
  • show less
    DOI: 10.3788/LOP202158.2217001 Cite this Article Set citation alerts
    Zhongfa Liu, Yizhe Yang, Yu Fang, Xiaojing Wu, Siwei Zhu, Yong Yang. Fusion of Cell Refractive Index and Bright-Field Micrographs Based on Convolutional Neural Networks[J]. Laser & Optoelectronics Progress, 2021, 58(22): 2217001 Copy Citation Text show less
    Schematic of graphene-based refractive index microscopy system
    Fig. 1. Schematic of graphene-based refractive index microscopy system
    Analysis of principle and microscopic images. (a) Schematic of the principle of probe beam scanning to measure the refractive index of the cell; (b)microscopic image of cell refractive index obtained in experiment; (c) experimentally obtained bright-field micrograph of the cell
    Fig. 2. Analysis of principle and microscopic images. (a) Schematic of the principle of probe beam scanning to measure the refractive index of the cell; (b)microscopic image of cell refractive index obtained in experiment; (c) experimentally obtained bright-field micrograph of the cell
    Fusion model FusionCNN
    Fig. 3. Fusion model FusionCNN
    The framework of FusionCNN algorithm
    Fig. 4. The framework of FusionCNN algorithm
    Refractive index micrographs and bright-field images of three cells. (a)--(c) Refractive index micrographs of cell; (d)--(f) the corresponding cells bright-field images
    Fig. 5. Refractive index micrographs and bright-field images of three cells. (a)--(c) Refractive index micrographs of cell; (d)--(f) the corresponding cells bright-field images
    Experimental results of fusion of refractive index micrographs and corresponding bright-field images of three groups of cells using GTF (gradient transfer fusion) method, WL (wavelet transform-based fusion) method, and FusionCNN (CNN algorithm-based fusion) method, respectively. (a)--(c) Original refractive index micrographs; (d)--(f) fusion results obtained using FusionCNN method; (g)--(i) fusion results obtained using GTF method; (j)--(l) fusion results obtained using WL method
    Fig. 6. Experimental results of fusion of refractive index micrographs and corresponding bright-field images of three groups of cells using GTF (gradient transfer fusion) method, WL (wavelet transform-based fusion) method, and FusionCNN (CNN algorithm-based fusion) method, respectively. (a)--(c) Original refractive index micrographs; (d)--(f) fusion results obtained using FusionCNN method; (g)--(i) fusion results obtained using GTF method; (j)--(l) fusion results obtained using WL method
    Fusion of high spatial resolution bright-field image or low spatial resolution bright-field image with refractive index microscopic image. (a) Fusion using 700 pixel×700 pixel bright-field image and 100 pixel×100 pixel refractive index microscopic image; (b) fusion of 100 pixel×100 pixel bright-field image and 100 pixel×100 pixel refractive index microscopic image
    Fig. 7. Fusion of high spatial resolution bright-field image or low spatial resolution bright-field image with refractive index microscopic image. (a) Fusion using 700 pixel×700 pixel bright-field image and 100 pixel×100 pixel refractive index microscopic image; (b) fusion of 100 pixel×100 pixel bright-field image and 100 pixel×100 pixel refractive index microscopic image
    Evaluation indicatorFormulaic expressionPhysical significance
    PSNR2552i=1Mj=1NF(i,j)-R(i,j)2Reflects the difference between two images at a specific pixel, the higher the peak signal-to-noise ratio, the closer it is to the ideal image and the better the result
    ENT-i=0L-1Pilog2(Pi)Reflects the amount of information in an image. The more information an image contains, the better the result
    AG1MNx=1My=1NΔfx2+Δfy2The larger the average gradient, the clearer the detail representation in the image
    Table 1. Objective evaluation indicators[21]
    Cell and fusion methodPSNRENTAG
    Cell 1FusionCNN24.71916.72510.0365
    GTF24.25056.55510.0367
    WL11.72056.38390.0283
    Cell 2FusionCNN25.77896.32780.0314
    GTF25.65256.12100.0303
    WL11.48846.04910.0024
    Cell 3FusionCNN26.15636.46380.0290
    GTF26.07566.38350.0272
    WL12.22166.45010.0206
    Table 2. Fusion performance comparison of different methods
    Zhongfa Liu, Yizhe Yang, Yu Fang, Xiaojing Wu, Siwei Zhu, Yong Yang. Fusion of Cell Refractive Index and Bright-Field Micrographs Based on Convolutional Neural Networks[J]. Laser & Optoelectronics Progress, 2021, 58(22): 2217001
    Download Citation