Obtaining more attribute information about the target is the goal the optical sensor constantly pursues. Polarization spectral imaging technology, which combines polarization attribute detection and traditional spectral imaging technology, can distinguish “different objects with the same spectrum”, and achieve “target highlighting” and “dynamic adjustment”. The shortcomings of the current polarization spectral imaging system include complex structure, large volume, channel crosstalk, and cumbersome multi-dimensional information extraction. In this paper, a compact polarization spectral imaging method based on Linear Variable Filter (LVF) and pixelated polarization modulation is proposed to solve the above problems. The work includes: under the constraints of high spectral resolution and short focal length, the double Gaussian structure is used as the initial optical structure, and the simulation and implementation of the optical system are carried out at the same time; The polarization modulation detector is coupled on the image plane to achieve simultaneous acquisition of spectral information and polarization information. In the laboratory darkroom, the optical index test of the system prototype developed based on the above technical route is carried out. The final index is as follows: working band: 430~880 nm, spatial resolution: 0.22 mrad, spectral resolution: 10 nm, synchronous acquisition of four polarization states, System transfer function: 0.547, polarization detection accuracy: 89.4%, total size of optomechanical system: 45 mm×45 mm×80 mm. Experiments were carried out outdoors, and the conclusion was that the monochromatic images of different polarization states of the central wavelength have obvious intensity changes; the multi-dimensional information extraction and fusion of the global image show that the characteristic spectral curves of different objects have obvious spectral differences. This method breaks through the shortcomings of the traditional polarization spectral imaging technology route and provides a new and important application method for the multi-dimensional information acquisition of polarization spectral imaging.