Raman spectroscopy, as a high sensitive, non-invasive vibrational molecular spectrocopy technique, plays a significant role in many fields such as pharmaceutical, biology and analytical chemistry etc. However, due to the weak Raman scattering intensity, the measured Raman signal is always contaminated by noise. Especially in the short exposure time, the SNR (signal to noise ratio) of collected Raman spectra is extremely low. Therefore, this paper proposed a signal reconstruction method based on matching pursuit algorithm, which is used to extract Raman signals from the low SNR Raman spectra. The method first finds the position of the characteristic peak on the average spectrum by threshold iterative method, and estimates the interval of the peak according to the location of the peak and peak interval, with a Gaussian density function to generate a dictionary. In the noise spectrum, according to the position and interval of the characteristic peak, it is divided into the signal interval and the non-signal interval. On the signal interval, the matching pursuit algorithm is used to reconstruct the Raman signal covered by noise. The algorithm not only can primely approximate the Raman signal which is covered in the noise, but also deducts the baseline in the procession of reconstructing the signal, and does not need any baseline correction further. The performances of the proposed algorithm and conventional algorithms were compared. The results show that the proposed algorithm can recover the Raman signals in the condition of low SNR. Different with the conventional de-noise algorithms, algorithm of this paper process the baselines and the random noises in Raman signals simultaneously, and the results have been proved good. So there is no need to use different algorithms to process the baselines and noises separately. Furthermore, in the aspect of algorithm, we creatively applied the matching pursuit algorithm to solve the sparse approximation of Raman signals.