With the rapid development of space remote sensing technology, the demand for space application of high-resolution infrared detectors increases. However, the refrigerator of this type of detector assembly is disturbed obviously during operation, which will affect the imaging quality of the detector. This disturbance needs to be suppressed to ensure the working state of the detector assembly on orbit. A super large-scale infrared detector assembly with a self-damping design structure is firstly introduced, and the effective research results are verified. Based on the obtained refrigerator disturbance test results, vibration isolation structures with different disturbance transmission paths are designed. Through simulation analysis and correction, the ground test verification is finally completed. This design can meet the needs of the detector component to achieve ultra-high-resolution disturbance suppression.