To enhance the robustness of size estimation for shaft parts in visual measurements, this paper proposes a fast and high-precision vision measurement method for shaft part geometric dimensions based on parallel line fitting. Parallel constraint improves the noise resistance of the proposed method and enhances the adaptability of the industrial measuring device to its environment. First, a region of interest (ROI) was quickly found based on template matching. Then, a parallel line equation was established using the Ransac algorithm for the interior points of the two edge lines of a measured object. Additionally, a nonlinear optimization was performed based on the least square method to obtain the number of pixels occupied by the measured object. In the simulation, the anti-noise performance of the proposed method was verified by comparing it with traditional algorithms. Finally, a telecentric measuring platform was built to measure the shaft neck size for a stepped shaft. The results showed the effectiveness and feasibility of the proposed method.