To carry out the precise measurement of large-scale complex workpieces, accurate calibration of the stereo vision measurement system has been becoming more and more important. This paper proposes a flexible and reliable camera calibration of stereo vision measurement system based on unit quaternions with optical reference bar, which has three small collinear infrared LED feature points and the lengths between these feature points have been precisely calibrated. By imaging the optical reference bar at different locations and orientations over the measurement volume, we can calibrate the stereo vision measurement systems. During the calibration process, we can automatically control the light intensity and optimize the exposure time to get uniform intensity profile of the image points at different distance and obtain higher signal-to-noise ratio. The experiment alresult proves that the proposed calibration method can obtain good results in the application, the maximum measurement error is 0.15 mm and the maximum standard deviation is 0.11 mm.