The presence of rain patterns in an image increases the difficulty of target detection and recognition. Rain patterns are the high-frequency parts of the image, which contain several image details. However, a major challenge is in removing the rain pattern while retaining useful details. Thus, we proposed a method based on a multiscale convolutional neural network developed using multilevel and multiattention mechanisms. To avoid the suboptimal effect of rain removal caused by preprocessing, we simulated the imaging process of objects in real scenes, improved the general image restoration model, enriched the network’s receptive field range, and accurately removed rain while enhancing contrast. Furthermore, we extract multiscale feature maps from the network branch using a multiconvolution feature jump connection to compensate for the loss of detailed information in the convolution process and fusion of different levels of feature information. Additionally, we combined attention to form multiple multiscale residual attention submodules to recalibrate the global information in the channel dimension, removing redundancy while enhancing useful information and the primary and advanced features were fused to learn the mapping relationship between the rain and no-rain maps. Considering that the real rain map has no corresponding rain-free map, we used a synthetic dataset for training, and used the synthetic dataset and real scene graphs for verification. The experimental results show that our proposed network achieved a good rain removal effect while retaining detailed information irrespective of the size and density of the rain pattern.