The characteristics of beat-frequency signal (BFS) and the detection performance of the frequency-modulated continuous-wave (FMCW) coherent lidar are influenced by speckle noise. In order to estimate the impact and improve the system′s property, the scattered field is simplified utilizing geometrical optics approximation. The impact model of BFS affected by speckle field is derived by Monte Carlo simulation, and the optical antenna′s aperture is optimized. The system experiment is carried out, and the results demonstrate that the BFS intensity has a negative exponential relationship with the ratio of the surface-roughness height to the square of light wavelength, and it decreases more rapidly with weak scattering surfaces. The experimental results are in good agreement with the theoretical analysis and numerical simulation results. After system optimization, the BFS with speckle noise can be detected effectively, and the system′s range and velocity measurement errors are less than 1 cm and 0.05 cm/s, respectively.