Because the existing three-point light pen measurement system using pespective-three-point(P3P) algorithm to solve an equation is prone to the problem of multiple solutions, this paper proposes a three-point light pen multi-solution elimination algorithm based on light field imaging. First, high-precision internal reference calibration is performed for the central aperture of the light field camera. Second, the image is acquired by the light field camera, and the camera coordinates of the two sets of light pen laser points in the extracted central subaperture image are obtained by using the P3P algorithm. The Gaussian imaging formula is used to calculate the depth values of the two sets of solutions, and then the regional window light field is refocused to the two sets of depth planes. Finally, the appropriate sharpness evaluation method is selected to compare the sharpness of the two images, and the final light pen pose matrix is obtained to solve the three-dimensional coordinates of the pen tip. In this paper, the moving distance of the precision three-axis translation stage is selected as the comparison benchmark. The experimental results show that the maximum position differences between the experimental measurement system and the three-axis translation stage in X, Y, and Z directions are 0.35, 0.40, and 0.54 mm, respectively. At the same time, the same positive results are obtained by using four-point re-projection, which proves the validity and accuracy of the proposed method.