By using the polarization characteristics of reflection mirror and cooperating with magneto-optical crystals, the rotatory double reflection structure can automatically compensate the polarization of photons. The polarization independent reflection structure can be used in the movable quantum key distribution system. The structure requires the polarizations of two mirrors to be absolutely the same and the rotatory angles of magneto-optical crystals to accurately reach 90°. The polarization error caused by mirror mismatch and rotatory angles deviating from 90° is analyzed. The result shows that if the difference between two magneto-optical crystals and two mirrors is kept small enough, good stability of polarization of output pulse can be kept.