In order to clarify the working mechanism of the image field modulated Fourier transform imaging spectrometer, we establish the theoretical model of an image field modulated interference imaging spectrometer by analyzing the phase modulation characteristics of the imaging field induced by a multi-micro-mirror. The numerical calculation shows that the panoramic image of the target scene can be reconstructed by image shearing and image splicing on the obtained interference image data cube. In addition, the spectral information of each target point in the scene can be recovered by fringe splicing and spectral demodulation on the sheared interference image unit. In order to verify the working principle of this instrument, we conduct the interference imaging scanning experiment of the target scene with the developed prototype and obtain the interference image data cube of the scene target. Through the edge detection and feature registration on each frame interference image, we realize the shearing of the interference image unit and the splicing of the panoramic image. Meanwhile, by means of fringe splicing, baseline correction, addressing apodization, and discrete Fourier transform, we recover the spectrum of the feature object. Finally, the recovered spectrum is optimized by means of non-uniform sampling correction and empirical mode decomposition, and the spectral performance is effectively improved.