Aiming at the shortage that tensor product wavelet only emphasizes the horizontal and vertical directions, the properties and characteristics of two-dimensional wavelet are investigated. A fast algorithm for two-dimensional four-channel non-separable wavelet and its mathematical proof are presented. The computation of the algorithm is analyzed, and the construction method of filter banks is discussed. Theoretical deduction and analysis show that the computation speed of this algorithm is faster than that of the non-sampling two-dimensional non-separable wavelet transform and two-dimensional fast Fourier transform. The decomposition experiment of non-separable wavelet is performed by constructing filter banks, and it’s compared with the tensor product wavelet. Results show that the non-separable wavelet transform can extract more comprehensive information of each direction when the image is filtered.