Author Affiliations
1College of Engineering, South China Agricultural University, Guangzhou , Guangdong 510642, China2Foshan -Zhongke Innovation Research Institute of Intelligent Agriculture and Robotics, Foshan, Guangdong 528251, Chinashow less
Fig. 1. Annotation effect of dataset samples. (a) Handle; (b) button of elevator; (c) button of light
Fig. 2. Workflow of this research
Fig. 3. Mask R-CNN architecture
Fig. 4. Occlusion caused by different viewpoints of two cameras
Fig. 5. Point cloud edge noise filtering algorithm based on depth statistics. (a) Instance segmentation edge error; (b) edge noise of point clouds; (c) denoising effect
Fig. 6. Point cloud registration result. (a) Point cloud from perspective one; (b) point cloud from perspective two; (c) merged point cloud
Fig. 7. Optimization effect of bounding box based on PCA. (a) Simple bounding box; (b) optimized bounding box; (c) optimization effect
Fig. 8. Long arm disinfection robot and control debugging experiment scene. (a) Long arm disinfection robot; (b) control debugging experiment scene
Fig. 9. Diagram of IoU calculation
Fig. 10. Example instance segment result predicted by Mask R-CNN
Fig. 11. Comparison of surface area of bounding box before and after optimization. (a) Handle; (b) button of elevator; (c) button of light
Fig. 12. Comparison of bounding box volume before and after optimization. (a) Handle; (b) button of elevator; (c) button of light
Configuration parameter | Value |
---|
Image per GPU | 1 | Batch size | 1 | Step per epoch | 1000 | Image minimum dimension | 800 | Image maximum dimension | 1024 | Epoch for training network heads | 40 | Learning rate for network heads | 0.001 | Epoch for training ResNet | 120 | Learning rate for training ResNet | 0.001 | Epoch for training all layers | 160 | Learning rate for training all layers | 0.0001 |
|
Table 1. Configuration list of network training parameters
Category | Number of images | AP | mAP |
---|
Ground truth | False positive | True positive |
---|
Handle | 91 | 3 | 86 | 0.945 | 0.968 | Button of elevator | 58 | 0 | 58 | 1.000 | Button of light | 77 | 5 | 77 | 0.997 |
|
Table 2. Object detection result
Category | IoU | Mean IoU |
---|
Handle | 0.873 | 0.879 | Button of elevator | 0.813 | Button of light | 0.933 |
|
Table 3. Mask R-CNN training results
Category | Optimization rate | Mean optimization rate |
---|
Handle | 0.270 | 0.292 | Button of elevator | 0.241 | Button of light | 0.364 |
|
Table 4. Optimization rate of bounding box surface area for various disinfection targets
Category | Optimization rate | Mean optimization rate |
---|
Handle | 0.279 | 0.288 | Button of elevator | 0.245 | Button of light | 0.372 |
|
Table 5. Optimization rate of bounding box volume for various disinfection targets