The use of medium-wave and long-wave infrared dual-band combination can effectively improve the performance of airborne photoelectric detection device. The design method using four mirrors and optical passive athermal technology is introduced. A dual-band and dual fields-of-view (FOVs) catadioptric optical system is designed. The system consists of two parts, i.e., the four-mirror telescope system and secondary imaging aggregation system, and uses optical passive thermal design. It meets the design requirements of lightweight, miniaturization, and athermalization. The optical system works in the 4.5~5.1 μm and 7.7~9.7 μm wavelength bands. It can achieve the zoom of 100 mm and 300 mm, with the zoom ratio of 3, F number of 2, and a compact structure and can meet 100% cold shield efficiency. The final optical design along with their modulation transfer function (MTF) is presented, showing excellent imaging performance in dual FOVs and dual-band at the temperature range between -60 ℃ and 70 ℃.