In order to improve the theoretical prediction accuracy of the image motion which is affected by coupling characteristics of the flywheel and the satellite, a numerical compensation algorithm for the stiffness and damping parameters of the on-board flywheel installation is proposed. Firstly, the mathematical model of the compensation algorithm is established. Based on the radial swaying mode of the flywheel and the linear whole-wave micro-vibration transfer function model, and by combining the measured data of disturbances of flywheel on the dedicated measuring platform with jitter-affected image motion data of flywheel installed on the satellite to construct an optimization function to compensate for the installation stiffness and damping parameters of the flywheel on the satellite. Secondly, the algorithm is used to compensate and estimate the stiffness and damping of the flywheel installation in a certain type of satellite. Finally, the theoretical prediction and comparison of the jitter-affected image motion before and after the parameter compensation are carried out. The comparison results show that the estimated image motion data after parameter compensation is closer to the measured image motion data than estimated image motion data after parameter compensation and the radial rocking mode natural frequency curve is more obvious, which proves that this parameter compensation algorithm is feasible. This study significantly reduces the analysis error caused by the inconsistent installation stiffness of the flywheel on the satellite and the installation stiffness of the test platform during the whole-wave micro-vibration analysis, and provides a new idea for solving the equivalent parameters of the coupling characteristics of the flywheel and the satellite.