We propose ResUNet+, an enhanced multiscale features residual U-shape network, to address issues in the extraction of small and irregular buildings from remote sensing images using the ResUNet, such as low segmentation accuracy and rough boundaries. Based on the ResUNet architecture, the squeeze and excitation module is used in the encoder to improve the network’s ability to learn effective features, and the atrous spatial pyramid pooling module is selected as the last layer of the encoding network to obtain context information of buildings at various scales. We evaluate the proposed ResUNet+ and compare it with SE-UNet, DeepLabv3+, DenseASPP, and ResUNet semantic segmentation networks on two commoly used public datasets: the WHU Aerial Imagery Dataset and INRIA Buildings Dataset. The results of the experiments show that ResUNet+ outperforms other networks in terms of pecision, recall, and F₁-score. The segmentation results also show that RseUNet+ excels at extracting buildings of various sizes and irregular shapes.