Raman imaging is a noninvasive, label-free spectral imaging technique that has been widely used in the biomedical field. However, the spontaneous Raman signals of most biological samples are weak. It takes a long time to obtain an image with a high signal-to-noise ratio, which seriously affects the spatial and temporal resolution of Raman imaging and hinders its application in fast dynamic systems. The multi-channel Raman imaging is one of the effective ways to solve this problem. The reconstruction of the full Raman spectrum is the key in this system, and the corresponding algorithm of reconstruction is needed to be developed. At present, the algorithms capable for spectral reconstruction are pseudo-inverse and Wiener estimation. Although these methods are simple and easy to be carried out, they are susceptible to nonlinear factors such as noise and vibration when applied to the system of multi-channel. On the other hand, the numbers of the training sample are relatively small, and the bad sample affects the reconstruction in the system of multi-channel. In order to solve those problems, we propose an algorithm based on sample optimization and principal component analysis (PCA). Firstly, the simulated narrow-band measurements of the training samples are calculated by using the spectral response function of the filter, and the full Raman spectra are reconstructed by Wiener estimation, and then the simulated narrow-band measurements of the reconstructed spectra are achieved. The sample gets Optimized by comparing the simulated narrow-band measurements of the sample and the reconstructed spectra. Second, the interference of nonlinear factors is reduced by introducing the polynomial regression and expanding the optimized narrow-band measurements. At last, the main information of training samples is extracted, and the calculation is reduced by using PCA, and the transform matrix is completed. At the same time, the normalization is introduced to realize the reconstruction of Raman spectra. In the experiment, the polymethyl methacrylate is selected as the experimental sample, and the Raman spectrum is reconstructed by pseudo-inverse, Wiener estimation and our algorithm. The root means square error is used to evaluate the accuracy of the reconstructed spectra. The result proves that our algorithm is significant. It provides theoretical support for the further application of Raman imaging technology in fast dynamic systems.