Aiming at the wide application of the visual coordinate measurement of the handheld target, an on-site two-step calibration method for the probe tip center of the planar target is proposed. The conversion relationship between the target coordinate system and the camera coordinate system is established by two sets of target images in different positions. Based on the position invariant principle, the object optimal function is established according to the image information of the first set target parallel to imaging plane with different angles. The Levenberg-Marquardt algorithm is employed to solve the optimal corrected value of the probe tip center along x and y directions in target coordinate. The error of the probe tip center along z direction is solved by using the image information of the second set target relative to the imaging plane of the camera with different angles. The correctness of the method is verified by simulation and experiment. The experiment results show that the repeatability accuracy of the probe tip center coordinates in x, y and z direction can reach 0.077 mm, 0.035 mm and 0.140 mm, respectively, which can effectively solve the problem of on-site calibration of the probe tip center of the handheld planar target.