• Laser & Optoelectronics Progress
  • Vol. 55, Issue 2, 021503 (2018)
fan Liu, Pengyuan Liu*, Junning Zhang, and Binbin Xu
Author Affiliations
  • Mechanical Engineering College, Shijiazhuang, Hebei 050003, China
  • show less
    DOI: 10.3788/LOP55.021503 Cite this Article Set citation alerts
    fan Liu, Pengyuan Liu, Junning Zhang, Binbin Xu. Joint Detection of RGB-D Images Based on Double Flow Convolutional Neural Network[J]. Laser & Optoelectronics Progress, 2018, 55(2): 021503 Copy Citation Text show less
    Convolution neural network structure
    Fig. 1. Convolution neural network structure
    Basic model of neuron
    Fig. 2. Basic model of neuron
    Image convolution process
    Fig. 3. Image convolution process
    Average pooling operation
    Fig. 4. Average pooling operation
    Backward propagation model
    Fig. 5. Backward propagation model
    Early fusion structure
    Fig. 6. Early fusion structure
    Late fusion structure
    Fig. 7. Late fusion structure
    Full connection layer fusion structure
    Fig. 8. Full connection layer fusion structure
    Convolution layer fusion structure
    Fig. 9. Convolution layer fusion structure
    Change curves of center position error and training loss function with the training steps
    Fig. 10. Change curves of center position error and training loss function with the training steps
    Detection results at different algorithms. (a) Detection based on RGB images; (b) joint detection of RGB-D based on late fusion; (c) joint detection of RGB-D based on convolution layer fusion
    Fig. 11. Detection results at different algorithms. (a) Detection based on RGB images; (b) joint detection of RGB-D based on late fusion; (c) joint detection of RGB-D based on convolution layer fusion
    RGB-accuracyD-accuracyRGB-weightD-weight
    Flashlight82.877.20.5180.482
    Coffee cup80.475.80.5140.486
    Cereal boxes83.278.60.5130.487
    Bowl78.475.10.5110.489
    Table 1. Fusion weight of different detection objects
    MethodCentralerrorAccuracyrate /%Successrate /%Detectiontime /s
    RGB image0.032481.275.40.228
    Depth image0.037176.771.90.177
    Early fusion0.029285.679.40.248
    Late fusion0.027787.181.30.325
    FC-fusion0.025888.382.20.306
    C-fusion0.023591.284.80.288
    Table 2. Detection results by different methods
    fan Liu, Pengyuan Liu, Junning Zhang, Binbin Xu. Joint Detection of RGB-D Images Based on Double Flow Convolutional Neural Network[J]. Laser & Optoelectronics Progress, 2018, 55(2): 021503
    Download Citation