High precision repetition rate stabilization is realized in an all polarization-maintaining nonlinear amplifying loop mirror (NALM) mode-locked Yb-fiber laser. An additional Er-fiber is added in the nonlinear loop to control the optical length by the resonance-enhanced nonlinearity modulation technique. Moreover, an integrated nonreciprocal device with linear phase shift effectively reduces the mode-locking threshold. By optimizing both the pump power from two laser diodes for mode locking and the nonlinear refractive index modulating, we can obtain the shortest pulse of 590 fs and repetition rate of 20.48 MHz, the peak-to-peak fluctuation of repetition rate is less than 0.4 mHz, and the corresponding standard deviation is 0.1 mHz.