Based on the least square method, the average surface rigid-body motion equation of optical elements is derived herein using the area-weighted average motion. By analyzing the effect of micro-vibration on the modulation transfer function (MTF) of an optical system, the relevant image motion and its corresponding optical axis rotation angle are determined. The data process for the frequency response analysis is performed using the average rigid-body motion equations. The micro-vibration magnitude of the photoelectric payload mounting surface is retrieved based on the demands of optical axis rotation angles of the system. The real-time MTF testing of the system is performed using the constant-frequency micro-vibration test. The results show that the error discrepancy between the analysis results and the test data is within 20%, which meets the requirements for engineering applications and verifies the correctness of the computational method.