Super-resolution single-image reconstruction is widely used in many fields; however, most existing algorithms extract more feature details by extending the depth and width of the convolutional neural network, which increases the computational complexity as well as number of model parameters. To solve these problems, a lightweight super-resolution network algorithm with adaptive residual attention is proposed herein. This algorithm first generates an attention feature map, which focuses on high-frequency location information by improving the coordinate attention network, and then connecting the improved adaptive residual attention information-extraction module and coordinate attention module in parallel to obtain more image details in the output feature information. The multiscale upsampling method was used to enable the trained network model (single training) to output multiple super-resolution images with different scales at once. Compared with the classical lightweight super-resolution algorithm, the fourfold reconstructed image obtained using the proposed algorithm has an average peak signal-to-noise ratio (PSNR) improvement of 1.06 dB on four common datasets and the number of model parameters is reduced by 59%. Further, the image obtained using the proposed algorithm is clearer and contains more high-frequency details.