In this work, we investigate the possibility of achieving nanojoule level pulse energy in an all-fiber Er-doped oscillator by using a graded index multimode fiber (GIMF) as the saturable absorber (SA). This GIMF-based SA demonstrates the desirable characteristics of high-power tolerance, large modulation depth of 29.6%, and small saturation fluence of $～7.19\times {10}^3\mathrm{\mu J}/{\mathrm{cm}}^2$, which contribute to the high-energy soliton generation. In the experiments, the oscillator generates stable ultrafast pulse trains with high pulse energy/average output power up to 13.65 nJ/212.4 mW in the anomalous regime and 6.25 nJ/72.5mW in the normal regime, which are among the highest energy/average output power values achieved by all-fiber Er lasers. The results obtained demonstrate that the GIMF-based SA can be used as an effective photonic device for high-energy wave-breaking free pulse generation.