Moving target detection is an important research direction of object detection, and it plays an important role in target recognition. The accuracy of traditional motion detection methods is low, which are unable to only detect the required moving target. In this study, deep convolutional neural network is introduced into the optical flow detection of moving target. In this method, a pair of images and optical flow fields of target are used as inputs of the network to adaptively study the target optical flow. Furthermore, through optimization of the expanding part of the network and the simplification of the network, and combined with many data augmentation technologies, the optical flow detection network of target object with both accuracy and real-time is designed. Experimental results show that the proposed method has better performance in the optical flow detection of moving target. SS-sp and CS-sp network are improved by about 5.0% compared to the original network on the precision and the runtime of the network is significantly reduced, which meet the requirements of real-time detection.