Aiming at the problems of low detection accuracy, low recall rate, and poor real-time performance of remote sensing images, a remote sensing aircraft detection algorithm based on GhostNet and CoT(Contextual Transformer) Multi-Branch Residual Network (MBRNet) is proposed. Learning from the YOLOv4 network model, MBRNet is adopted as new backbone network to reduce the problem of gradient disappearance and makes up for the lack of global feature calculation capabilities of CNN. In order to reduce the problem of small target loss, multi-directional feature extraction and fusion are introduced into the backbone and PANet. The idea is to realize full complementation of information between high and low feature layers and between feature layers of the same scale.The proposed algorithm has an accuracy of 97.64% and a recall rate of 89.11% on RSOD and LEVIR data sets in the circumstance of complex background, overexposure and dense targets.