The low-distortion, large-image-field telescope is the key to the precise image fusion and registration of wide-band hyperspectral imaging system. Based on the analytical methods and vector aberration theory, the distortion of the off-axis three-mirror optical system is analyzed. A new method to design the low-distortion off-axis three-mirror optical system is proposed. A coaxial three-mirror optical system is designed by analytical methods and then the original system is made off-axis. Then the system is optimized according to analysis based on the vector aberration theory. An optical system is designed which is characterized by the system focal length of 1000 mm, cross track field of 5.6°, along track field of 1° and F number of 5. The maximum distortion measured by Zemax software is only 0.0628%. This method can be applied to design telescopes not only in the large-field and high-resolution imaging spectrometer system, but also in the occasions that demand low distortion.