To address the problem of high resource consumption and difficulty of hardware transplantation involved in utilizing deep convolutional networks for real-time detection of remote sensing building, a semantic segmentation network based on the mixed method of binary and floating-point parameters, i.e., mixed binary U-shape network (MBU-Net), is proposed. To compress the model size, the weights of a float U-shape network (FU-Net) are binarized. The output layer weights that account for a small number of parameters are replaced by floating-point type parameters to resolve the poor detection accuracy and low training speed in a global binary network. Experiments using the QuickBird satellite remote sensing dataset show that the pixel accuracy of MBU-Net is 82.33% and the harmonic average of the recall rate and accuracy rate (F1 score ) is 73.15%. Compared with the FU-Net,the MBU-Net can ensure the detection accuracy. The size of model is greatly compressed, the detection speed is increased by 6.29 times, and the power consumption is reduced to 37.78%, further demonstrating that the MBU-Net is superior to other similar methods (Deeplab and ENet). This finding has important practical engineering value for the real-time detection of remote sensing buildings.