To address the urgent need for ultra-large aperture and lightweight high-resolution optical payloads in the field of high-orbit military monitoring, an image inversion and quality enhancement method was proposed for large-caliber diffractive imaging systems with diffractive optical elements used as the primary mirror. First, an image inversion model was established for the diffraction imaging system based on the regularization and iso-halo block theories. Here, the multi-regularization and multi-constraint algorithm was adopted for theoretical model optimization, and two bottleneck problems were in turn solved to a certain extent. One was the image blurring caused by both the large-caliber optical system aberration and the diffraction effect of the main mirror; the other was the large spatial variation of the point spread function. Nonlinear transformation and wavelet threshold filtering were then used to formulate a contrast enhancement method to address the low signal-to-noise ratio and low contrast problems that derive from low diffraction efficiency and non-design level sub-background clutter. Finally, experiments and a simulation were conducted to test the proposed method. Experimental results show that the structural similarity of degraded and original images exceeds 08, the Signal to Noise Ratio(SNR) is increased by 10% and fidelity is more than 80% with CLE of 30. The proposed method improves image quality, including image clarity and contrast as well as the suppression of noise and background radiation. The proposed method thus has great theoretical significance and engineering application value for the practical application of ultra-large aperture thin-film diffraction systems.