Author Affiliations
Key Laboratory of Photoelectronic Imaging Technology and System of Ministry of Education, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, Chinashow less
Fig. 1. Construction and pipeline of the algorithm
Fig. 2. Depthwise separable convolution
Fig. 3. Squeeze-and-excitation module
[22] Fig. 4. Hand gesture recognition network based on MobileNetv3-SSDLite
Fig. 5. Neural network structure before and after the embedded optimization
Fig. 6. The chosen hand gestures
Fig. 7. Selecting images for hand gesture dataset
Fig. 8. Network loss in training process
Fig. 9. NVIDIA Jetson TX2 embedded processor developer kit
Fig. 10. Part of the hand gesture recognition results
network structure | params/Mbyte | MACs/106 | ImageNet
accuracy/%
| VGG16 | 13.8 | 15300 | 71.5 | MobieNetv1 | 4.2 | 569 | 70.6 | MobileNetv2 | 3.4 | 300 | 72.0 | MobileNetv3 | 5.4 | 219 | 75.2 |
|
Table 1. MobileNet series comparison to VGG16
extra layers | shape | layer 1 | $39 \times 39 \times 512$ | layer 2 | $19 \times 19 \times 1024$ | layer 3 | $10 \times 10 \times 512$ | layer 4 | $5 \times 5 \times 256$ | layer 5 | $3 \times 3 \times 256$ | layer 6 | $1 \times 1 \times 256$ |
|
Table 2. Extra feature map layers for object detection
network structure | params/Mbyte | MACs/106 | mAP/% | SSD | 14.8 | 1250 | 19.3 | SSDLite | 2.1 | 350 | 22.2 |
|
Table 3. SSDLite detection head comparison to SSD
hand gesture | accuracy/% | 0 | 99.64 | 1 | 100.00 | 3 | 99.51 | 4 | 99.22 | 5 | 99.69 | average | 99.61 |
|
Table 4. Recognition results of hand gestures
scenarios | average accuracy/% | multiple hand gestures | 96 | complicated background | 64 | low light intensity | 72 |
|
Table 5. Recognition results of hand gestureson various scenarios
algorithm | params/Mbyte | MACs/106 | frame rate/(frame/s) | mean accuracy/% | VGG16-SSD | 24.3 | 30654 | 2 | 91.75 | MobieNetv1-SSD | 7.2 | 1299 | 12 | 93.98 | MobileNetv1-SSDLite | 4.1 | 1130 | 16 | 93.86 | MobileNetv2-SSDLite | 3.1 | 656 | 36 | 91.01 | MobileNetv3-SSDLite | 2.2 | 526 | 58 | 99.61 |
|
Table 6. Comparison of different hand gesture recognition algorithms.