A broad-spectral imager integrates three detectors which include a visible multi-spectral detector, a short-wave/middle-wave infrared detector and a long-wave infrared detector, and it can provide coverage over the broadest spectral range of more than 10 spectrums in the same type of domestic remote sensors so far, which involves more complex systems and more difficult design challenges. In this paper, a design of data control and processing system of multi-detector was given to solve these challenging problems of the broad-spectral imager such as synchronization control of multi-detector, mass data storage and integrated processing, imaging parameters adjustment of detectors which include gain, stage, integral time and others on orbit, etc. Field programmable gate array(FPGA) was used as a key cell of control and processing in the system hardware; and optocoupler and low-voltage differential signal(LVDS) devices were used to transmit signals to other systems; By means of modularized conceptual design, the software was designed from top to bottom with very-high-speed integrated circuit hardware description language(VHDL); Finally, verification of the design were performed with simulation tools and hardware simulator, and the results show its correctness and effectiveness. In practical engineering application, the system has comprehensive functions, flexible interface, high reliability and expansibility.