In this paper, we propose a method of forgery numeral handwriting detection based on convolution neural network (CNN). It provides an intelligent solution for forgery document detection. The experiment convened 50 volunteers and collected image samples of six types of forged handwritings and normal handwriting with 50 different brand pens, and established a total of more than 7200 sample data. Then, we designed a new CNN for forgery numeral handwriting detection called FNNet by introducing Fire Module structure based on AlexNet. We replaced the partial 3×3 convolution kernel with 1×1 convolution kernel and performed convolution layer assembly to detect forged samples. The experimental results show that the average test accuracy of FNNet in the six types of handwritten forgery numbers is 98.36%, which is 3.01 percentage higher than that of AlexNet. The proposed method is superior to traditional feature classifiers; it provides a new method for forged handwriting detection.