Optical rectification effect is one of the effective ways to generate terahertz(THz) wave, and the generation of high-power THz wave requires femtosecond laser with high intensity to excite optical rectification crystals. However, the crystal will be damaged if the intensity of femtosecond laser is too high, seriously affecting the output of high-power THz wave. Therefore, it is of great importance to study the damage thresholds of optical rectification crystals for both theoretically and practically. In this paper, the interaction mechanism between the femtosecond laser and the nonlinear optical rectification crystals was analyzed, and the prediction model of damage thresholds of optical rectification crystals under femtosecond laser was built up. Then on this basis, the variation of damage thresholds of LiNbO3, ZnTe and ZnSe crystals with the pulse duration of femtosecond laser was compared and analyzed. The results show that, the photoionization dominates in the whole process when the pulse duration of femtosecond laser is relatively short, while the avalanche ionization gradually plays an important role with the increase of pulse duration. Avalanche ionization rate and photoionization rate are related closely with the forbidden band width of the crystal. Consequently, the greater the forbidden band widths of optical rectification crystals, the higher the damage thresholds are. The damage thresholds of LiNbO3 are higher than those of ZnTe and ZnSe crystals, which is more suitable for the generation of high-power THz wave.