In many complex environments, intelligent manufacturing technology using robots faces many challenges. There are rapid response to changes in the workspace, false perception, error of noise and control, and even resistance to disturbances of the robot itself. The grasp detection of target in the real working scene is an important step in robot workflow, which involves grasp positioning and attitude estimation (Fig. 1). Due to the complexity of robot grasp detection, in recent years extensive research has committed to using deep learning method to achieve grasp detection. The objective is to improve the accuracy, timeliness and stability of robot grasp detection. Deep learning methods for real-time robot detection and grasp are divided into three main categories. They are sliding window-based, bounding box-based, and pixel-level methods. The sliding window-based method is effective for grasp detection. But the large number of iterations and the slow response speed make it difficult to meet the real-time requirements. Bounding box-based methods have a positive effect on target detection. But the structure of the model is complex and a large quantity of parameters should be learnt. In the present study, we report a robot grasp detection method based on a stable pixel-level lightweight network to reduce the number of parameters. We hope that our method can save more computation and memory during robot grasp detection. And it can ensure the high accuracy, timeliness and stability under complex conditions.

Firstly, an enhanced dataset is created by using random cropping, scaling, and rotation on the Cornell dataset (Fig. 2). Then, we build a lightweight network based on U-net (Fig. 5). Residual blocks are added to increase the number of extracted feature layers in the network. They can suppress the gradient vanishing and dimensional error (Fig. 6). At the same time, instance normalization (IN) is used to design the normalization layer of the residual block and each convolutional block in the network (Fig. 3). It will increase the stability of the detected image instance and accelerates the model convergence (Fig. 6). In addition, the network integrates the idea from the feature pyramid network (FPN) structure (Fig. 4). We connect the top-down feature map with the bottom-up feature map through the horizontal connection layer. The former has strong semantics, low resolution and easy target recognition. The latter has weak semantics, high resolution and easy target localization. Meanwhile, the multi-dimensional information is integrated to improve the localization ability and semantic information of the output feature map (Fig. 5). Finally, in order to avoid the problem of gradient explosion and outlier interference, our study uses the Huber loss function to analyze the calculation results.

In this study, all comparison methods uniformly use the Cornell dataset. And we use different intersection-over-union (IOU) standards to evaluate the network performance (Table 1). As the IOU standard becomes more strict, our network changes smoothly (Fig. 7). When the IOU is 0.5, the accuracy of our network can still reach 80.9%. Therefore, our network performance is more stable and competitive in robustness to complex environments. When the IOU is 0.25, the network model designed in this study has 94.4% accuracy, 40.8 frame/s speed and 478900 total parameter quantity (Table 2). The results show that our method can achieve high accuracy and speed when the total parameter quantity keeps small. The model is tested using random objects in the Cornell dataset (Fig. 8). And we also perform real-time detection of unknown objects (Fig. 9). The experiment shows that the network designed can output the grasp frame with accurate positioning and posture information for objects with different properties, including objects partially in the figure, translucent objects, reflective objects, bifurcated objects, irregularly shaped objects, and even transparent objects.

This study is based on simultaneously achieving high accuracy and timeliness of robot grasp detection, reducing the number of system parameters and ensuring the stability of the network. Referring to the pixel-based U-net lightweight network model, we design a robot grasp detection method based on stable lightweight network. Firstly, the quantitative results show that the simulated curve is stable and the total parameter quantity of the method is 478900. Our method can maintain more than 80% accuracy when the IOU is less than 0.5, outperforming the existing methods. At the same time, when the IOU is 0.25, it has a high accuracy of 94.4% and a speed of 40.8 frame/s. It is rational that this method can meet the current requirements for the accuracy, timeliness and stability of robot grasp detection. Moreover, the qualitative results show that the network model designed in this study is effective on objects with different properties. Especially, our method is not disturbed by the situation that the current depth camera cannot accurately measure transparent objects. Finally, it is shown that adding IN and FPN to the lightweight network can effectively improve the performance of robot grasp detection.