Galvanometer scanning systems are widely used in various fields such as laser rapid prototyping, laser precision marking, and laser scanning measurement for their high speed and precision. However, system measurement errors are induced because of the introduction of a galvanometer system. In view of this, firstly, a linear laser one-dimensional galvanometer scanning system is built and calibrated by the double-checkerboard calibration method. Then, the error model of the system is established and the measurement error in the scanning process is theoretically analyzed. Finally, a compensation method based on the table look-up method is proposed for the galvanometer rotation error. The experimental results show that when the measuring working distance is about 250 mm, after the error compensation, the square root error of center distance decreases from 0.733 mm to 0.061 mm and the standard deviation from 0.200 mm to 0.060 mm. It follows that this method can significantly improve the measurement accuracy and robustness of the system.