Real-time detection and recognition of target is a hot issue in the field of interferential infrared imaging spectrometer, and the real-time spectrum recovery of the spectrometer is the prerequisite to solve the problem effectively. The interference pattern signal was obtained by using the visible light camera to simulate the interferometer, and the real-time spectrum recovery system of interferential infrared imaging spectrometer was designed by using the FPGA chip, which had high speed and large capacity. The system is mainly composed of interferogram data processing module, real-time spectrum recovery module and spectrum display module, running in line way and output the target′s real-time spectrum information. It has the advantages of high speed, small size and the algorithm is easy to upgrade. The system can establish a good technical foundation for the real-time detection and recognition of the target by the spectrometer.