The purpose of this study is to develop a method for automatically identifying weeds in a rapeseed field. We propose a weed-recognition method based on a Faster R-CNN (region-convolution neural network) deep network and use the deep network model of the COCO dataset for migration training. First, by obtaining images of rapeseed and weed samples under natural environment, the Faster R-CNN deep network model is utilized to share the convolution characteristics and the results of three feature extraction networks: VGG-16, ResNet-50, and ResNet-101, are compared. At the same time, the method is also compared with a single shot multibox detector (SSD) deep network model, which includes the three identical feature extraction networks. The results show that the Faster R-CNN deep network model based on VGG-16 has obvious advantages in rapeseed and weed target recognition. The accuracy of target recognition and recall rate of the rapeseed and weeds are 83.90% and 78.86%, respectively, whereas the F1 value is 81.30%. The proposed deep learning method can effectively and rapidly identity rapeseed and weed targets, providing a reference for further research into multi-type weed target recognition.