In order to solve the diffraction problem in the spectrum simulation of the star simulator， a method to eliminate the diffraction effect of the Digital Mirror Device （DMD） is proposed. A scalar diffraction model is established， and the DMD is equivalent to a two-dimensional blazed grating. Based on the grating diffraction theory， the factors affecting the diffraction energy level are searched for， and the coupling relationship between the incident angle and the diffraction effect is deduced. When the light beam is incident along the diagonal of the DMD and the height angle is equal to the angle of height， the diffraction efficiency is maximized， and the diffraction efficiency is increased by nearly 12%. At this time， the diffraction loss energy is the least， and the diffraction effect when the DMD reflects is minimized. Through ZEMAX Optimal design of Czerny-Turner optical structure to correct system aberrations. The optimization results show that the spectral simulation accuracy is better than 5% in the 500~900 nm spectral range， and the spectral resolution of the full spectral range is better than 5 nm， which can effectively improve the ground calibration accuracy of the star sensor.