In order to realize the active control of nonreciprocal optical transmission, a structure based on graphene photonic crystals and magneto-optical semiconductor materials is proposed. By adding external magnetic field on the magneto-optical semiconductor, the time-reversal symmetry is broken which contributes to the obtainment of nonreciprocal magnetic Tamm plasmon polaritons (MTPP). The influences of structural parameters, external magnetic field and graphene chemical potential on the MTPP are studied by analyzing the transmission spectra under different conditions. The results show that the active control of the unidirectional wave and the flexible switch between one-way forward and one-way backward transmission can be realized by modulating the external magnetic field and graphene chemical potentials. This structure and its modulation method have promising applications in optical diodes and optical switches.