In the application of ultrasonic infrared thermographic technology, it is usually necessary to extract features from infrared thermographic images based on artificial experience and then adopt a pattern recognition method to classify the cracks. The identification and positioning process of the cracks is complicated, and the recognition rate is low. Therefore, a method of crack detection and recognition in ultrasonic infrared thermal images based on convolutional neural network technology is proposed in this paper. Its feature is that the features can be directly learned from the ultrasonic infrared image to realize the classification of infrared thermal images containing cracks. Thesis through the research experiment of metal plate specimen of the crack in and do not contain infrared thermal images, the convolutional neural network model is established for whether the image contains crack classification, the results show that the parameter optimized convolution neural network model for ultrasonic infrared thermal images of crack classification accuracy rate reached 98.7%.