When the intelligent algorithm is used to analyze the complex geological samples with serious overlapping spectral peaks, there are some problems such as big calculation, large error of weak peaks, convergence to local minimum or non-convergence. Because of its good convergence, the quantum genetic algorithm can decompose overlapping peaks in X-ray fluorescence spectra. A method of overlapping peak decomposition based on the GMM-ER model and quantum genetic algorithm with multi-fitness function is proposed. The overlapping peak model (GMM-ER) is first introduced based on K-series and L-Series of element characteristic X-ray. Then, based on the physical characteristics of the X-ray fluorescence spectrum, a multi-fitness function is introduced into the traditional quantum genetic algorithm. The simulated spectra are generated by the characteristic X-rays of Mn, Fe, Co and Ni. Then, based on the GMM-ER model, the simulated spectra are analyzed 10 times by traditional quantum genetic algorithm and improved multi-fitness quantum genetic algorithm, respectively. The experimental results show that the average decomposition accuracy of overlapping peaks is improved by 32.1%, and the optimal decomposition accuracy is improved by 73.9%. Using the improved algorithm, the decomposition accuracy of elements with a low content ratio is greatly improved, and the relative error range of element decomposition is reduced by 64.5% under the optimal decomposition accuracy. Moreover, the convergence speed of the improved algorithm is faster than that of the traditional algorithm. This method is suitable for the decomposition of seriously overlapped peaks and has a high resolution for weak peaks.