We propose an improved convolution neural network based on LeNet-5 to address the problems of large computation and over-fitting in the full convolution neural network based method for eliminating noise of seismic data. The network of the proposed method consists of two convolution layers, two pooling layers, and one full output layer, in addition to the input layer. By using the experimental selection method of minimum error, the parameters of the first convolution layer and the pooling layer in the single-layer convolution network are determined. Then the parameters of the second convolution layer and the pooling layer are determined based on the parameters of the first layer. Finally, 12000 seismic data with size of 32×32 are used as inputs to train LeNet-5, and 1000 seismic data with the same size and signal-to-noise ratio are used for testing the system. Experiments on pre-stack and post-stack seismic data from Marousi2 model demonstrate that the proposed method has good denoising effect for horizontal and inclined in-phase axis seismic data. Compared with the singular value decomposition algorithm, BP (back propagation) algorithm, and algorithm in Ref. [9], the proposed method has better denoising effect.