A cross-polarization converter based on hollow butterfly graphene is proposed. The continuous plasmon resonance at the edge of the butterfly shape can be excited due to the surface plasmons can be supported by graphene. The proposed polarization converter can achieve excellent broadband polarization conversion performances by using a simpler structure. Numerical simulations show that when the Fermi level and relaxation time of graphene are 0.4 eV and 1.0 ps, the polarization converter can achieve a polarization conversion rate greater than 90% in the frequency range of 0.947‒1.452 THz , the relative bandwidth is 42.1%, and the center frequency is 1.2 THz. Moreover, the proposed polarization converter shows strong robustness to the incident angle of the incident terahertz wave. The polarization conversion rate can be maintained above 80% in a wide frequency range when the incident angle is increased to 40°. The high-efficiency working bandwidth and polarization conversion rate of the polarization converter can also be modulated by adjusting the Fermi level and relaxation time of graphene. This design method can have huge application potential in the field of photonics.