In this study, we design an optical system for a visible-infrared (VNIR-SWIR) Offner imaging spectrometer for the space exploration of asteroids. The operation waveband of the proposed system covers from 0.4 μm to 2.7 μm, and the F number is 6 and 3 for the visible/near-infrared (VNIR) band (0.4--1.0 μm) and the short wave infrared (SWIR) band (1.0--2.7 μm), respectively. Subsequent to the initial structure development and optimization, spot diameters in the spot diagrams of the proposed optical system are observed to be less than one pixel, and the minimum modulation transfer function (MTF) is 0.51 at the Nyquist frequency. To improve the spectral response to fulfill the broadband spectral imaging requirement, the convex grating is divided into two regions with different grating periods. The inner region works in the VNIR band, while the other works in the SWIR band. The prototype of the imaging spectrometer optical system is developed and its spectral performance is characterized. Results show that the performance of the proposed prototype is satisfactory and can meet the requirements.