The high peak-average power ratio (PAPR) is a major factor affecting the performance of the optical orthogonal frequency division multiplexing (OOFDM) system. We put forward a novel coherent optical orthogonal frequency division multiplexing (CO-OFDM) system in which we use the fractional Fourier transform (FRFT) to replace the traditional fast Fourier transform (FFT). Focus is put on the analysis of the principle that FRFT can reduce PAPR. The optical system simulation experiments show that the algorithm has obvious inhibitory effect on PAPR, but this inhibition cannot be improved infinitely: when the order is less than 0.001, the effect is no longer changed significantly.