The pulsed 1064 nm fiber laser is used to clean the thermal oxide layer of FV520B steel. The effects of processing parameters including laser power, pulse frequency, cleaning times, and substrate roughness, on the cleaning effectiveness of thermal oxide layer and the surface roughness after cleaning are investigated. The results show that for the pretreated samples (the roughness is approximately 0.503 μm) by polishing with 200# abrasive paper, the surface roughness remains almost unchanged after cleaning in the laser power range of 40-120 W, while the roughness decreases with increasing power in the range of 120-200 W. When the above pretreated samples are cleaned under laser power of 120 W and pulse frequency of 20 kHz, the surface roughness presents an obvious decrease (dropped by nearly 40.8 %) after laser cleaning for two times, and the roughness does not basically decrease with the increase of cleaning times. In addition, under the same laser cleaning parameters, the cleaning effectiveness of oxide layer becomes lower and the roughness does not exhibit any obvious change with the decrease in roughness of pretreated samples.