Total integrated scatter (TIS) is used to measure the root mean square (RMS) roughness of the fused quartz substrates cleaned by Kaufmann ion source and the K9 glass substrates cleaned by ultrasonic and End-hall ion source, respectively, to study the influence of the above cleaning methods on the surface micro-roughness of substrates. It is found that the RMS roughness of the K9 glass substrates cleaned by ultrasonic under different conditions all increases due to the damnification of the substrate surfaces, while the surface RMS roughness of the samples cleaned by End-hall ion source and Kaufmann ion source is significantly affected by the experiment parameters, such as the beam current, the cleaning time, and the energy of the ion beam. The RMS roughness of the samples can be decreased under appropriate experiment conditions.