Signal extraction will be affected by noise and baseline distortion in most kinds of the spectrum. If the influence of noise and baseline distortion is not considered in spectrum signal extraction, the accuracy of signal extraction will be seriously decreased. Therefore, it is necessary to eliminate the influence of noise and baseline distortion before signal extraction. However, most signal extraction algorithms' procedure is to extract the whole baseline first and then extract the signal, which makes it difficult to guarantee the extraction accuracy of the baseline. A spectrum signal detection and extraction algorithm (SSD algorithm) based on saliency and statistical characteristics was proposed because the presence of signals always causes the statistical characteristics of the signal region to be different from the background. Firstly, the signal's saliency at different scales is calculated, and the detected significant signal points are taken as candidate signal points. Secondly, the pseudo-signal points in the candidate signal points are removed based on the signal characteristic that the signal should satisfy. Finally, the quadratic polynomial is used to fit the candidate signal region's baseline to remove the false signal areas and realize the final signal extraction. Many experiments were run to verify the performance of the SSD algorithm. Firstly, gaussian signal and rectangular signal were simulated under different baseline types and signal-to-noise ratio (SNR). Then different algorithms were compared, such as the AirPLS algorithm, Wavelet algorithm and DoG algorithm, on the extraction results. Simulation experiment results show that: SSD algorithm was better than compared algorithms.The signal extraction results of the SSD algorithm were not affected by the signal type and baseline distortion type and were not affected by SNR when SNR is greater than 40. Its accuracy, stability, and dispersion were good, while the other algorithms are only applicable to a certain type of baseline distortion. From the overall extraction results, the mean value of the absolute error of the SSD algorithm is only 8.71% of the AirPLS algorithm, 3.52% of the Wavelet algorithm, and 2.01% of the DoG algorithm; the root means square of the absolute error is also only 13.08% of the AirPLS algorithm, 5.45% of Wavelet algorithm, 3.11% of DoG algorithm. Therefore, the SSD algorithm proposed in this paper has good comprehensive performance in extracting signals and can accurately extract signals under different SNR and baseline distortion.