The coherent detection sensitivity can be directly reflected by mixing efficiency in free-space coherent optical communication system. Expressions of the variation in mixing efficiency with the radial error of optical axis, mismatching angle, axial defocusing and atmospheric turbulence intensity are derived respectively, and simulation analysis is carried out. Results show that mixing efficiency can reach the maximum value of 0.8145 when we choose the waist radius and relative aperture matched with wavelength. The optical axis radial error of signal light and local oscillator light has the greatest influence on the mixing efficiency and relative aperture, followed by the mismatching angle and axial defocusing. The above three factors are limited to 0-4 μm, 0-0.05 rad and 0-200 μm respectively. Mixing efficiency is larger than 0.55 when the 1550 nm light wave is chosen as the communication wavelength, the waist radius is 3.4λ and relative aperture is 0.21. The atmospheric turbulence effect can be effectively suppressed with a small aperture receiving antenna under the mid-strong turbulent condition.