In infrared images, when the traditional histogram equalizes the image, the detail pixels are easily immerged by the background pixels, resulting in the image being too bright and too dark. Based on this situation, an adaptive inverse histogram equalization algorithm was proposed in this paper. The algorithm enhanced image details by inverse statistics, adaptive selection threshold and segmentation mapping. Compared with the traditional histogram equalization algorithm, the inverse histogram equalization algorithm significantly improve the image visual effect in different gray level distributions and enhance the details of different areas of the image to different degrees. Moreover, under the premise of achieving better image processing effects, this algorithm can still guarantee real-time performance and high efficiency by optimizing calculation methods, and is suitable for FPGA hardware transplantation.