Soil is an important carrier of the human living environments. Therefore, the problem of soil heavy metal pollution has always attracted attention. With the development of remote sensing technology, much progress has been made in the area of hyperspectral remote sensing, which is used for the study of the heavy metal content of soil. However, this method works basically in accordance with spectral absorption features, as well as the content of soil organic matter, iron and clay minerals, when retrieving the heavy metal content of soil. It has been found to be unable to distinguish the slight differences in the soil heavy metal pollution spectra. In this study, a potting soil pollution experiment with different concentrations of copper (Cu) and lead (Pb) was used to obtain the potting soil spectral curve, as well as the water and organic matter content of the soil under different concentrations of Cu and Pb stress. The purpose of the experiment was to put forward a type of second-order differential Gabor expansion method for the detection of the slight differences between the soil’s spectral curves under different concentrations of Cu and Pb stress. Firstly, based on the second-order difference method, the soil spectrawere converted into sparse spectra by the method, then the sparse spectrum of soil and Gabor expansion theory were combined to detect the weak differences of heavy metal stress spectra in different concentration soils in the frequency domain. Therefore, instead of studying the content of soil heavy metal solely by the spectral reflectance information, this method performed time-frequency analysis on the spectral information of soil heavy metal stress and finally achieved the purpose of detecting the instantaneous spectrum of soil heavy metal pollution. The results showed that the Cu-contaminated potting soil spectra displayed major differences in the scale distribution of the second-order differential Gabor expansion coefficient when compared with the Pb-contaminated potting soil spectra. In addition, the scale of second-order differential Gabor expansion coefficient of the Cu-contaminated soil spectra ranged from 1 800th to 3 600th items sparse, the scale of second-order differential Gabor expansion coefficient of the Pb-contaminated soil spectra ranged from 3 200th to 3 600th items coarctate. By utilizing a second-order differential Gabor expansion method, the detected results of the Cu and Pb pollution in the soil were found to be closely related to soil’s Cu and Pb content, water content and organic matter. As a result of the different Cu and Pb content in the soil, as well as the organic matter content and water content, the second-order differential Gabor expansion spectra of the soil’s Cu and Pb pollution displayed different scale distributions. According to the correlation analysis results, the soil’s Cu and Pb pollution were divided into three grades respectively: Cu(50)~Cu(300), Cu(400)~Cu(800), Cu(1 000) or more; lowerthan Pb(50), Pb(100)~Pb(300), Pb(400)~Pb(1 200).