Aiming at the noise disturbance of unwrapping phases of control points in the camera-projector calibration system, a random sample consensus (RANSAC) based plane fitting method is proposed to filter the phase noise in this paper. Different from the classical least squares method using all data, the points with noise will not be used to fit the plane in the proposed RANSAC method, which improves the accuracy of plane fitting. The proposed method suits for any two-dimensional (2D) calibration patterns, such as checkerboard or black dots with white background, which improves the flexibility of camera-projector system calibration.