The unmixing of traditional Chinese painting pigments has always been an essential content in the study of ancient painting pigments, and the fiber optics reflectance spectra (FORS) is a standard method for nondestructive detection of pigment category. In this paper, through the CCD (charge-coupled device) optical fiber spectroscopy system, the classification of traditional Chinese painting pigments in terms of spectral lines was discussed. The diffuse reflectance and absorption spectra of two kinds of organic plant pigments gamboge and rouge in different mass ratios were detected, and the diffuse reflectance spectra of inorganic mineral pigments in different color series were obtained. The characteristic spectral peaks of a single pigment and different mixed pigments were analyzed, and the mass ratio of each pigment was obtained by the linear unmixing method of the whole band using multiple linear regression (MLR) and first derivative spectroscopy (FDS). Through experimental and theoretical analysis, the diffuse reflection spectra of gamboge and rouge are all S-type. The first derivative spectra of the mixed pigment have two characteristic peaks at 536 and 649 nm. MLR is suitable for unmixing the mixed pigment diffuse reflectance spectrum and shows a certain linear rule, but it cannot be precisely dis mixed. However, the absorption spectrum of the mixed pigment has a utterly linear relationship with the monochromatic spectrum, and the unmixing error is about 5%. The diffuse reflectance spectra of inorganic mineral pigments are S-type (mineral yellow and ochre) and bell-type (malachite and azurite). Firstly, for the diffuse reflection spectrum of S-type (mineral yellow) and S-type (ochre), the first derivative spectrum of ochre presents an obvious phenomenon of “triple-peak”. Meanwhile, the first derivative spectrum of mixed pigments presents a new characteristic peak at 534 nm. However, it cannot form a more accurate linear model due to the different weight factors of unmixing of different pigments. Secondly, for the mixed spectrum of S-type (ochre) and bell-type (malachite), it is necessary to use multiple linear regression and derivative spectroscopy to determine the basic trend of their mixing ratio jointly. Because of different lines, the reflection spectrum of mixed pigments has only one crossing point in the range of 400~800 nm. Finally, through the characteristic the peak positions of bell-type (malachite) and bell-type (azurite) pigment mixing spectra, the characteristics of pigment mixing ratio can be acquired. With the change of the mixing ratio, the characteristic peak position of the reflection spectrum showed a significant lateral movement at 457~524 nm, and the peak intensity of the mixed pigment spectrum decreased significantly.