A robust least squares integration surface reconstruction algorithm that exploits the relative angle difference is proposed for a topography measurement device. The cost function of the classical least squares integration technique is rewritten based on relative angle difference to avoid accumulation of measurement errors. The space and time complexities are O(N2) and O(N3), respectively. Simulation results demonstrate that the robustness of the proposed algorithm is significantly better than that of Zernike’s wavefront reconstruction method and spline-based least squares integration method. Experiments show that the algorithm can be applied to topography measurement with large aperture angle difference method.