Given the large number of species of flies and their individual complex characteristics, recognizing a particular type of fly has proved to be time consuming and, for the most part, inaccurate. In this paper, a method for the facial recognition of a fly using deep learning technologies was proposed, specifically a Convolutional Neural Network (CNN), and its related face recognition algorithms. Initially, a multi-task convolutional neural network was utilized and optimized for the image alignment process. Thus, depth-wise separable convolutions were applied to reduce the number of calculation parameters as well as the image preprocessing time. Next, we combined the rough extraction of contour features and fine extraction of specific parts to derive more abundant feature information. The convolution and pooling layers were harnessed to elicit contour eigenvalues of the image, while Inception-ResNet and Reduction networks were administered simultaneously to obtain eigenvalues of specific parts. Finally, the above methods were coalesced to enhance the accuracy and comprehensibility of the resultant feature information for network training. Experimental results show that the mean average precision of the proposed method is 94.03%. When compared with other network training methods, this method not only improves the computational efficiency but also ensures high accuracy.