Age change is one of the main reasons that affect the performance of face recognition. In order to solve the problem of low face recognition rate caused by the change of age, a cross-age face recognition model (CA-CNN) based on deep learning is proposed for cross-age face recognition. First, the overall face features are extracted from the face image by the convolutional neural network; then, an efficient convolutional attention module is proposed to obtain age features from overall face features, and combined with multi-layer perceptrons and multi-task supervised learning, the overall face features are non-linearly decomposed into age features and identity features; finally, for better distinguish between identity features and age features, an approved batch kernel canonical correlation analysis module is put forward to analyze the correlation between the decomposed identity features and age features. After the training of adversarial learning, the correlation is minimized and cross-age face recognition is realized. The proposed model achieves the recognition accuracy up to 99.03% of rank-1 on the MORPH Album 2 dataset, and the face verification of equal error rate of 9.8% on the CALFM dataset, which indicates the effectiveness of the proposed model.