Since blurred image caused by defocus is of low clarity, image restoration is necessary. For traditional methods, circular disk or Gaussian function is adopted to approximate point spread function (PSF) caused by defocus, and the restored image is not ideal. Therefore, a method using Hartmann-Shack wavefront sensor to detect the wavefront distorted by defocus, calculating the PSF of the optical system with the obtained wavefront, and adopting the Richardson-Lucy algorithm to deblur the degraded image is proposed. An experimental system is built, from which both the blurred image and the corresponding wavefront information are collected. Clear restored images are obtained. Objective image quality assessment methods are used to evaluate the degraded and restored images. In addition, the restored images are also compared with the results obtained by traditional methods. Experimental results show that the proposed method can accurately reconstruct the PSF, and the quality of the image is efficiently improved.