In recent years, the single image super-resolution methods based on deep learning have made remarkable achievements. However, these methods focus researches on the image spatial domain, ignoring the significance of information in high frequency in image frequent domain, resulting in a relatively smooth image. A single image super-resolution method combining wavelet transform with deep network is proposed, which takes the advantages of extracting the details by wavelet transform. First, the image is decomposed into a sub-band in low frequency and three sub-bands of different directions in high frequency by wavelet transform, then the low resolution image and sub-bands in high frequency are regarded as the input of the deep network. Second, the existing deep network is improved by simplifying the network, decreasing the number of convolution layers to reduce network burden, and modifying the network channels. Finally, the super-resolution image is obtained by inversely wavelet transforming. The proposed method is tested on the open test datasets, and compared with other state-of-the-art methods. The experimental results demonstrate that the proposed method works well in subjective visual effects and objective evaluation indexes.