Infrared dim moving target detection technology is a hot and difficult research area in computer vision. To deal with the challenges of target detection with airborne in high dynamic air to ground background, such as dynamic scene change, large background interference intensity and unknown target motion law, a novel incremental inertial navigation information assisted air to ground infrared dim moving target detection algorithm was proposed. To solve the drift error problem of traditional inertial navigation information prediction, the concept of incremental inertial navigation information was put forward. The location prediction model of incremental inertial navigation information (LPI) was designed and the accurate prediction of the target point was achieved. A moving target detection framework was constructed based on inertial navigation information assistance and background difference, which corrected the images under different positions and attitudes by LPI. The cross correlation matching algorithm based on mountain climbing method was introduced to calculate the translation parameters, and Gaussian weighting was used to estimate the background. The dim moving target could be detected by adaptive threshold segmentation. The simulation experiments verified the effectiveness and accuracy of the proposed detection algorithm.