Rain and fog weather seriously affects the quality of outdoor images. In this paper, a new method of defogging based on manifold particle filtering is proposed to solve the problem of edge artifacts. By optimizing the atmospheric transmissivity, the accurate transmissivity is obtained, and the problem of edge artifacts in the depth of field is solved. Aiming at rain marks and unclear problems in removing rain and fog, this paper proposes a method that optimizes the attentive generative adversarial network. By combining the Gaussian model with the generative adversarial network, the background interference is removed, and the accuracy of separation of the background layer from the rain line is improved. At the same time, the manifold particle filter fog removal module is added to the generative adversarial network to recover the clear image without rain and fog. The rain and fog images in the natural scene are used for testing, and qualitative and quantitative analyses are conducted. Experimental results show that compared with the existing rain-removal algorithm, the proposed algorithm can remove the rain line in image effectively, and the details are more abundant. At the same time, the addition of the fog removal module significantly improves the image clarity and the objective index.