Graphene is attractive for the great potential in microwave, millimeter wave, terahertz wave and other fields due to their excellent electronical properties. In this paper, a rectangular metallic waveguide integrated with graphene for phase and amplitude modulator is designed in the millimeter and sub-terahertz waves frequencies. Through adjusting the surface impedance of graphene, the amplitude and phase of electromagnetic wave propagation in the waveguide can be controlled dynamically. Meanwhile, the influence of the length and position of the integrated graphene sheet on the transmission and reflection coefficients of electromagnetic wave in the waveguide is analyzed, as well as the effect of graphene chemical potential on the transmission and reflection of electromagnetic wave in waveguide. The results show that the reflection coefficient, transmission coefficient and transmission phase modulation range of the modulator can be customized by adjusting the length of graphene sheet and its position in rectangular metal waveguide, which meets the needs of device level applications.