Photonic-aided millimeter-wave communication combines the advantages of optical fiber and wireless communications, and it has several applications in the next generation of wireless access network systems. Geometric shaping (GS) technology can effectively overcome the signal damage caused by the transmission channel's nonlinear effect through optimising the Euclidean distance between the constellations and can effectively improve system transmission capacity and spectrum efficiency without increasing the system hardware costs. The transmission performance of 8/16-Gbaud 80-GHz vector millimeter-wave signal in the modulation formats of 8PSK/GS-8PSK and 8QAM/GS-8QAM is studied using an optical link scheme that uses a single push-pull modulator to generate vector millimeter-wave signal, combined with GS technology. The results show that when compared with traditional modulation formats, the geometrically shaped signal has a considerable improvement in the bit error rate and optical power under back-to-back and optical fiber transmission conditions. After transmitting an 8-Gbaud GS-8PSK signal through 150 km optical fiber, the optical power is increased by approximately 1 dBm; after transmitting a 16-Gbaud GS-8QAM modulated signal through back-to-back and 80 km optical fiber, the optical power is increased by approximately 0.5 dBm.