A mathematical model and calibration method is proposed for column two-dimensional portable vision metrology. The analysis of the measuring principle and system components is given. The overall mathematical model in line with measuring principle is established based on the exercise of independent positioning and camera imaging model. In the process of calibration, firstly, the calibration of geometric errors such as the positioning error is completed. Then, the image of round hole is evenly distributed in the image plane of the camera by moving the camera. At last, the calibration of camera imaging model parameters is achieved using nonlinear least squares fitting method, according to the target workbench position, pixel coordinates and the amount of the corresponding moving mechanism moves. Results show that the average error of measurement is 1.8 μm by modeling and calibration. The modeling and calibration method of portable vision metrology can effectively improve the system accuracy.