The viscosity of the polishing slurry is an important parameter during the fluid jet polishing (FJP) of optical elements. Polishing slurry viscosity in the FJP of hard and brittle optical components has not been extensively investigated. This study therefore investigates the influence of viscosity on the material removal function in the FJP process. The physical models of continuous-phase, discrete-phase, and erosion processes are also proposed for FJP. Pathlines of abrasive particles with different viscosities are calculated, and the variation rule of impact velocity vector of abrasive particles with viscosity is analyzed. Using the polishing slurries with different viscosities but the same mass fraction, the effect of viscosity on material removal function is obtained by combining the static mining-spot experiment with plastic-wear theory for a BK7 optical glass workpiece. The influence of viscosity on the material removal function and the influence on the surface roughness after uniform polishing are evaluated. The results indicate that as the slurry viscosity increases, the form of the material removal function and the material removal range remain the same, whereas the material removal depth decreases exponentially and the surface roughness of the optical elements is improved. Results of this study expand the knowledge of FJP material removal mechanisms and have theoretical significance regarding the control of polishing slurry viscosity.