Recent years, although the super-resolution reconstruction technology based on neural network has developed rapidly, there are still some shortcomings, such as difficult to find the appropriate size of convolution kernel, and slow convergence speed caused by too deep network layers. In this paper, a model which can extract features at multiple scales and contains multi-residual structure is proposed. Low-resolution image is input to the network, through serial multi-scale residual blocks, extracted and concatenated features at multiple scales in each block, after residual structure the image outputs to the next block, after all blocks, builds residual again, and finally outputs high-resolution image through sub-pixel convolution. The experimental results show that the proposal of multi-residual structure makes faster convergence, and the multi-scale structure extracts image features better to make the image excel other mainstream algorithms in whether subjective or objective measurement.