A new kernel anisotropic diffusion noise removal algorithm is proposed, through anisotropic diffusion and kernel method, to denoise linear crack on the light rail beam surface, and preferable results are obtained. On the basis of anisotropic diffusion, an enhancement operator which promotes the weak crack edge is added, and according to the characteristics of noise uniformly distributing in the multi-dimensional space, the low-dimensional data is promoted to high-dimensional space. Except for denoising in the kernel space, average absolute difference value of automatic diffusion termination criterion is introduced to enhance the efficiency of diffusion. A different edge enhanced weight is choosed to discuss the appropriate weight range. This method has been applied to noise removal of low signal-to-noise ratio track surface beam linear crack of 0.4 mm width. Compared with median filter and traditional P-M anisotropic diffusion, the kernel anisotropic diffusion outperforms them for denoised result and signal-to-noise ratio.