According to the high precision testing requirements on aiming at the baseline variation of infrared sight, a measuring method of infrared sight zero momentum based on camera pose adaptive mathematical model is proposed. A new adaptive algorithm of camera pose is designed. A camera pose correction model is established, which is applied to zero momentum measuring system, solving a problem of zero momentum measuring error caused by camera tilting. The pinhole targets and sighting divide image are located by gravity method, and sub-pixel subdivision of edge points is achieved by Zernike orthogonal moments. An experimental system is constructed. The measurement accuracy is superior to 0.02 mil (1 mil=0.254 cm) which is verified by experiment. The result meets the high precision testing requirement of infrared sight baseline variation.