The confocal displacement sensing system employs a small-aperture dispersive objective lens, achieving an extended working distance and overcoming limitations imposed by axial dispersion size. To meet the application requirements of small hole inner diameter or internal defect detection, a small-diameter dispersive objective based on a Gradient Index (GRIN) lens is designed. Through the study and correction of optical aberrations, the performance of the spectral confocal displacement sensing system has been optimized. GRIN lenses possess multiple degrees of refractive index parameters, offering an effect equivalent to homogeneous optical elements with complex shapes, high process demands, and elevated costs. They feature simple geometrical shapes, lightweight structures, compact sizes, excellent optical performance, and are conducive to optical integration. To analyze the impact of optical aberrations of GRIN dispersive objective on peak wavelength extraction and achieve performance optimization of the spectral confocal displacement sensing system based on GRIN dispersive objective, this paper investigates the influence of optical aberrations of GRIN dispersive objective on peak wavelength extraction and establishes an optical aberration fitting algorithm. Initially, considering the optical properties of the GRIN dispersive lens, the study utilizes the wavefront aberration equation in Fourier optics. It combines the refractive index variation of the GRIN dispersive lens with different wavelengths and curvature radii. Physical models for spherical aberration, astigmatism, and coma of the GRIN dispersive lens are established and analyzed within the spectral confocal vertical sampling sequence. An optimized distribution function for the aberrations of the GRIN dispersive lens is then developed, establishing relationships between various aberrations and wavelength distributions. Subsequently, various monochromatic aberrations and combined aberrations are simulated to analyze their impact on the distribution of peak wavelengths. Three fitting algorithms are employed to extract peak wavelengths under different aberrations, revealing the wavelength shift caused by optical aberrations of the GRIN dispersive lens. Finally, Gaussian fitting, Zernike polynomial fitting, and sinc² function fitting are applied to perform data fitting and analyze fitting errors for the axial response signals under different aberration scenarios. The results indicate that spherical aberration, astigmatism, and combined aberrations can cause axial response peak wavelength shifts. When the spherical aberration is 1, the peak wavelength shifts by 6.28 nm. For spherical aberration greater than 1, a double peak appears, and the larger the spherical aberration, the greater the peak wavelength shift. The impact of astigmatism on peak wavelength shift is smaller than that of spherical aberration. Combined aberrations have the greatest effect on peak wavelength, leading to the simultaneous elevation of sidelobes for three peaks, causing a significant spectral confocal position deviation or system error. The influence of off-axis aberration can be ignored. Finally, using three different methods to fit and analyze axial response signals under the influence of various aberrations, the results indicate that, in the case of optical aberrations in the GRIN dispersive lens axial response peak wavelength extraction, Gaussian fitting improves the accuracy of peak wavelength extraction. However, it's fitting performance for sidelobes is relatively poorer. In comparison to Gaussian and Zernike polynomial fitting, sinc² function fitting not only provides a good fit for the main peak but also yields better fitting results for the sidelobes than Gaussian and Zernike polynomial fitting. The dispersive lens is the central component of the spectral confocal displacement sensing system. The system's measurement precision is closely related to the resolution of the dispersive lens. The accuracy of measurements and the measurement range depend on the magnitude of the axial dispersion of the dispersive lens. Optical aberrations of the dispersive lens affect the axial distribution of the focal wavelength, causing interference in the collected spectral response data, consequently impacting the system's measurement performance. The research findings are of reference significance for establishing error correction and compensation algorithms for peak wavelength extraction affected by aberrations, further enhancing the measurement performance of spectral confocal systems.