To make varied-line-space holographic grating which in applied in the extreme ultraviolet band (50~150 nm), a new optimization algorithm applied to spherical wavefront system is proposed, named the improved local optimization algorithm. According to the request of instruments, expected varied-line-space holographic grating density function is built. Based on this density function, the merit function is given, of which characteristic is transforming four merit functions to one merit function which has multivariate and nonlinear constrains. Using the improved local algorithm, the error of groove density coefficient is reduced and the resolution of gratings is improved due to the weighted, the diversity of the initial value, the limitation of the recording parameters values range and the constraint of groove density coefficients. Compared with the conventional local optimization algorithm, using the improved local optimization algorithm makes absolute error between design and expected groove density in 0~0.02 line/mm range. And the order of magnitude is improved and the resolution increases from 4000 to above 17000. Results show that by adopting the suited optimization algorithm, varied-line-space plane grating with a high resolution can be made in the spherical wavefront system.