A calibration method suitable for a laser probe in on-machine laser scanning measurement was proposed to eliminate the measuring errors from installation pose. A prototype system for the on-machine laser scanning measurement was built and a model of the laser probe moved with a machine tool was established. The center of a standard ball was fitted by scanning the ball surface with multi-perspective, and a linear algorithm to solve the installation pose parameters of the laser probe was given to overcome the heavy computation and un-robustness of the existing nonlinear algorithms. The influence of the machine tool movement error on the measurement was also analyzed and an error model was established and a compensation method for the machine tool error was proposed. The experimental results show when the standard ball with a known diameter was measured, its diameter error measured by the laser probe at different angles is less than 0.05 mm, and the position error of the standard ball center is reduced by 83% with the compensation method. The results illustrate the feasibility of the calibration method as well as the validity of the error model and the compensation method.