Poor generalization ability of soil nitrogen content prediction models based on spectroscopy is the bottleneck of its actual application in agriculture production. However, the deep learning model shows strong potential for generalization because of its automatic feature extraction and excellent nonlinear expression. In this paper, a spectral prediction model of soil nitrogen content combining the auto-encoder and convolutional neural network (Encoder-CNN) was proposed, the influence of model structure and parameters on model performance was explored, and its generalization ability was investigated. After researching the previous references and analyzing the correlation between wavelengths and soil nitrogen content, 180 wavelengths with the highest correlation were selected and taken as the input of the Encoder-CNN model. The output of the Encoder-CNN model was the soil nitrogen content. The Encoder-CNN model first used the auto-encoder to reduce the dimension of 180 wavelengths and then predicted the soil nitrogen content by its convolutional neural network. Two network structures were designed. Each network structure had two different parameter settings. Therefore, four models were used to explore the effects of structure and parameters of the Encoder-CNN soil nitrogen content spectral prediction model on modeling performance. Encoder CNN models were trained by the LUCAS data set. According to the 3σ principle, 20 791 data were obtained from LUCAS and then divided into a training set (18 711) and test set (2 080). The results were analyzed and discussed, and several conclusions were achieved in this research. The reproduction effect of the automatic encoder reached the best when the number of neurons in the last hidden layer was set to 30 with the same number of hidden layers as the others; the more hidden layers, the better the reproduction effect. As for the prediction part based on CNN, increasing the number of convolution kernels, especially 1×1 convolution kernels, could improve the prediction performance and reliability. By adding pooling layer in CNN, the model's prediction accuracy was improved to more than 0.90. The model's performance could also be improved by increasing the number of neurons in the full junction layer. The Encoder-CNN model built by the LUCAS data set was migrated to the Heilongjiang data set, and the generalization ability of the model was observed and evaluated. The prediction accuracy of the model, which was trained 100 times by the Heilongjiang data set, could reach more than 0.90. When the number of iterations was set to 900, the model's prediction accuracy could be as high as 0.98. The results showed that the proposed Encoder-CNN spectral prediction model of soil nitrogen content had good generalization ability, and it could be used to detect soil nitrogen content after the model migration process.