A 40 GHz radio-over-fiber (ROF) system is investigated experimentally to generate optical millimeter wave with optical orthogonal frequency division multiplexing (OFDM) signal using an optical phase modulator or an optical intensity modulator, and the transmission performances of optical millimeter waves are compared. In the central station, the 2.5 Gb/s OFDM analog data and a 20 GHz radio frequency (RF) clock are mixed to drive an optical phase modulator or an optical intensity modulator to realize the double-sideband (DSB) modulation, generated optical millimeter wave with OFDM signal, transmitted to the base station over standard single-mode fiber(SSMF). In the base station, the optical millimeter wave signals are converted to electrical signals after optical-electrical conversion, and the down-converted OFDM signals are detected after mixed with the local oscillators, and the optical carrier filtered out by a fibber Bragg grating (FBG) can be used to realize wavelength reuse. Effect of different peak-to-average power ratios on bit error rate (BER) is demonstrated experimentally. The results show that under the high optical power, after transmission over 50 km SSMF, the power penalty for the optical millimeter wave generated by an optical phase modulator and an optical intensity modulator are less than 0.5 dB and more than 1 dB, respectively. It demonstrates that the fiber transmission quality of optical millimeter wave with OFDM signal generation using an optical phase modulator is better than an optical intensity modulator.