Expeditious calibration of vision measurement system with field coordinate system is fundamental and crucial, especially for movable visual tracking and measurement system. The calibrating principle of coordinate transformation was elaborated based on their characteristic, and mathematical model of the algorithm was proposed to calibrate the visual measurement system with field coordinate system. Two control points that parallel the axis in field coordinate were used as target. The calibrating results were computed based on azimuth and pitch angle of the visual measurement system when the line-of-sight collimated the control points. Simulation analysis and real data experiment results show that the angle error is 0.03°, and length error is 0.52%. The proposed algorithm is practical, convenient and wieldy to movable vision measurement coordinate system expeditious calibration.