To reduce the system bit error rate and increase the system rate of the radio of fiber (RoF) system without increasing the average optical power, this study proposes a W-band orthogonal frequency division multiplexing-radio of fiber (OFDM-RoF) system based on probability shaping (PS). The theory and operating rules of PS are analyzed theoretically. The OFDM-RoF system is developed on the simulation platform. In this system, the PS is used to adjust the transmission probability of each constellation point of 16 quadrature amplitude modulation (16QAM), and the 80 GHz PS-16QAM-OFDM millimeter-wave signal is generated jointly with the OFDM-RoF system. Results after the signals are transmitted 0, 10, and 20 km through an optical fiber show that when the symbol rate is 2.5 GBaud and lg R_BE= -9(R_BE is bit error tate), compared with the traditional 16QAM/OFDM-RoF system, the transmitting optical power of the PS-16QAM/OFDM-RoF system is reduced by 1 dBm, and its system performance is better. However, when the symbol rate is 10 GBaud, the constellation diagram of the 16QAM/OFDM-RoF system using PS diverges in the outer circle, but the constellation points in the inner circle are concentrated; its system performance is good.