Due to the geometric distortion of fisheye images, the existing person detection algorithms based on fisheye images have the problems of low detection accuracy and high computational complexity in post-processing.A rotation-aware person detection algorithm was proposed to solve the problems. First, the algorithm adopted an anchor-free network structure and used heatmap to predict the center point of the bounding box, there was no need to apply non-maximum suppression on the bounding boxes during post-processing which avoids the calculation of intersection over union(IoU) between rotated bounding boxes. Then,a Gaussian kernel function with angle and scale adaptation was adopted to fit the center distribution of person with distortions,which greatly reduced the interference of background features,and balanced the difference of person with different sizes under fisheye images during the bounding boxes regression. Finally, the Angle-IoU(AIoU) was designed to combine both IoU loss and Ln-norm loss, indicator function was used to deal with inconsistent regression between IoU loss and Ln-norm of angle regular term. The proposed algorithm was verified on public datasets, experimental results show that the algorithm has achieved the state-of-art performance with an average mAP of 51.33%, detection frame rate reaches 49 fps, which is 139% higher than the detection algorithm with anchor-based network structure, the comprehensive performance of the algorithm is better than other existing person detection algorithms in overhead fisheye images.