Using UAV aerial photography to collect pavement images can effectively improve the efficiency of road health detection. However, due to the influence of aerial photography angle and sunlight change, the long shadow produced by UAV aerial photography image will cover up the pavement damage information and affect the accuracy of damage detection. To address this issue, a shadow removal algorithm for aerial photography of pavement based on Generative Adversarial Networks (GAN) is proposed. The algorithm introduces a multi-scale feature extraction module in the generative network to enhance the capability of image information feature extraction. At the same time, depthwise separable convolution is used in the structure of the discriminator network, which effectively reduces the sensitivity of the model to non-shadow areas and improves the discriminant effect of the discriminant network. In addition, an aerial photography of pavement shadow dataset under different pavements and lighting conditions is constructed to improve the generalization ability and robustness of the model. Experimental results show that the shadow-free result images obtained by the algorithm have several no-reference image quality assessment indexes improved, which can improve the accuracy and integrity of pavement damage detection and identification.