The existing deep learning based SAR-assisted cloud removal methods do not take full into account the texture and spectral information of the optical images, which results in blurring and spectral loss. In this paper, we constructed a data set for SAR-assisted cloud removal based on the Sentinel-1 and Sentinel-2 satellite images in Yuhang District of Hangzhou. In addition, we established a conditional generative adversarial network (cGAN) based model by fully considering the details, texture, and color information of optical remote sensing images, achieving information recovery and reconstruction in the case of optical images covered by thin clouds, fog, and thick clouds. The results show that the proposed method outperforms other methods in SAR-assisted cloud removal.