In order to improve the visual effect of underwater images, an underwater image restoration algorithm based on red-dark channel prior (RDCP) and inverse filtering is proposed. Firstly, the Jaffe-McGlamery underwater optical imaging model is simplified. On this basis, the RDCP is used to eliminate the foggy appearance of images resulting from backward scattering during the imaging process. Secondly, considering the mathematic relation between the transmission map of each channel and optical transfer function, inverse filtering is applied to remove the forward scattering component. Finally, the proposed algorithm adopts linear stretch based on Gaussian distribution to improve image contrast. The proposed algorithm and several main underwater image processing algorithms are employed in processing underwater images captured in various underwater environments, and the information entropy and other objective evaluation factors are calculated. The experimental results prove that the proposed algorithm has superiority for balancing the color, contrast and saturation of the images, and the visual effects are more similar to images captured in natural settings.