In order to solve the problem that the detection of thin clouds and broken clouds is very difficult due to the changeable cloud shapes in the research of cloud detection in RGB color remote sensing images, a U-shaped network based on multi-scale feature extraction (MS-UNet) is proposed. Firstly, a multi-scale module is proposed in order to obtain a larger receptive field while retaining more semantic information of the image. Secondly, the FReLU (Funnel Rectified Linear Unit) activation function is introduced in the first group of convolutions to obtain more spatial information. Finally, further feature extraction is performed after down-sampling, and in the up-sampling pixel recovery, the missing information is completed by jump layers, and the deep semantic features of the cloud are combined with the shallow detail features to achieve better cloud segmentation. Experimental results show that this method can effectively segment thin clouds and broken clouds. Compared with UNet, MF-CNN, SegNet, DeepLabV3_ResNet50, and DeepLabV3_ResNet101 networks, the overall accuracy is increased by 0.075, 0.065, 0.070, 0.013, and 0.005, respectively.