In the process of wet-process phosphoric acid, the impurities in the mineral have great effects on the crystallization of calcium sulfate and restrict the utilization of phosphogypsum. Choosing the typical low-grade phosphorus rock in Hubei province of China as the material, the types and contents of main impurities in phosphorus rock and phosphogypsum have been analyzed through X-ray photoelectron spectroscopy, scanning electron microscopy, and EDS. The main changes of the impurities have been evolved in wet-process phosphoric acid. The results show that fluorapatite and quartz are the main phases in phosphorus rock. The highest impurity components are silicon, aluminum, fluorine, and magnesium. Silicon exists in silica and calcium silicate, and fluorine is all in fluorapatite, and aluminum exists in different kinds of aluminosilicate, and most of magnesium exists as MgF2. Calcium reacts with sulfuric acid to form gypsum through acidolysis reaction. Silicon mainly remains in solid phase. Most of aluminum, fluorine, and magnesium decompose into liquid phosphoric acid. Silica still remains in phosphogypsum, and calcium silicate dissolves in acid solution. The compound comprised of aluminum, silicon and phosphorus forms in acidolysis process. Fluorine is mainly in silicofluoride. Most of MgF2 dissolve into solution, and remaining part of Mg in phosphogypsum mainly exists as magnesium silicate. Through the research of occurrence of impurities, the system change may be better acknowledged in the process, and it can provide a basis for the investigation and control of phosphogypsum crystallization.