The transmission performance of orbital angular momentum(OAM) beam and Gaussian beam in atmospheric turbulent channel was studied and compared experimentally. The beam spreading, pointing error, power jitter and bit error rate of the two beams were measured respectively, the OAM beam loaded with modulated signals and the Gaussian beam were as carrier. The experimental results show that in the atmospheric turbulence channel, compared with Gaussian beam, beam spreading ratio of the OAM beams reduces by 10.5%, variance of power jitter falls 0.13, dispersion circle diameter of the pointing error decreases by 30.4%, and more focused on the distribution of the center of the beam axis. The lowest detection sensitivity of the OAM beam carrier system reaches -28.97 dBm, which improved 2.5 dB compared with that of Gaussian beam. The experimental results verify that OAM beam is less affected by turbulence than Gaussian beam in atmospheric turbulence channel, and the deterioration of beam quality of OAM beam is much less than Gaussian beam with the increase of turbulence intensity. The results of the experiment provide a reference for the development and application of free space laser communication in atmospheric turbulent channel.