For the target recognition of infrared imagery, a method was proposed via the combination of convolutional neural network (CNN) and joint sparse representation (JSR). CNN learned the deep features of the infrared target imagery, which described the multi-layer properties of the target. Different layers of deep features described the target charateristics from differnt aspects, so they can well complement each other. The joint use of multi-layer deep features could provide more valid information for target recognition. During the classification, JSR was employed to represent the multi-level deep feature vectors and the inner correlations among different features was used to improve the overall representation precision. Therefore, JSR not only made use of individual deep features but also considered their inner correlations. According to the outs from JSR, the target label of the input sample was determined based on the minimum error. The experiments were conducted based on mid-wave infrared (MWIR) dataset under the conditions of original test samples, noise test samples, and small training set. Simultaneously, the proposed method was compared with four previous methods. According to the experimental results, the proposed method achieves better performance under the three conditions, validating its potential in infrared imagery target recognition.