A new algorithm of system calibration based on depth constraint was presented to improve the calibration accuracy of large scale planar part vision measurement system. Firstly, an algorithm of extracting the round marks of calibration plate was proposed. Sub-pixel contours of marks were extracted by edge detection algorithm based on adaptive threshold and polynomial fitting, then the center coordinates can be computed by ellipse fitting method. Then, on the basis of analyzing camera distortion model, the optimal solution of internal parameters and plate poses were obtained using least square method. Finally, the pose of measuring plane was transformed by depth information as a correction term between parts surface and calibration plate. The dimensions could be calculated using imaging principle and the intersection point of the line and the part surface. Measurement system was designed and experiments were carried out. The result shows that the calibration back-projection error is less than 0.02 pixel and the precision of the system is up to 0.05 mm in the range of 1×0.75 m².