The output of the infrared imager is heteropical and dim, so the sharpening enhancement algorithm of the infrared image has the great significance. The effectiveness of the algorithm of infrared image combined the edge has been discussed in the software simulation, on that basis, the algorithm has a few improvements. The effectiveness of the algorithm in the sharpening enhancement about the hardware real-time image is mainly discussed. The FPGA + SRAM hardware structure is designed, the realization method of the algorithm in the hardware system is discussed, the effect of the algorithm realized on the hardware platform is analyzed, and the implementation effects between the software and hardware are compared. The image contour is strengthened after the sharpening algorithm and the objective is identified easily.